{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 点击率预估"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "点击率预估用来判断一条广告被用户点击的概率，对每次广告的点击做出预测，把用户最有可能点击的广告找出来，是广告技术最重要的算法之一。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据集下载"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"criteo.jpg\"\\>\n",
    "这次我们使用Kaggle上的[`Display Advertising Challenge`](https://www.kaggle.com/c/criteo-display-ad-challenge/)挑战的criteo数据集。\n",
    "\n",
    "下载数据集请在终端输入下面命令(脚本文件路径：./data/download.sh)："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "wget --no-check-certificate https://s3-eu-west-1.amazonaws.com/criteo-labs/dac.tar.gz\n",
    "\n",
    "tar zxf dac.tar.gz\n",
    "\n",
    "rm -f dac.tar.gz\n",
    "\n",
    "mkdir raw\n",
    "\n",
    "mv ./*.txt raw/\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "解压缩以后，train.txt文件11.7G，test.txt文件1.35G。\n",
    "\n",
    "数据量太大了，我们只使用前100万条数据。\n",
    "\n",
    "head -n 1000000 test.txt > test_sub100w.txt\n",
    "\n",
    "head -n 1000000 train.txt > train_sub100w.txt\n",
    "\n",
    "然后将文件名重新命名为train.txt和test.txt，文件位置不变。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data fields"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Label \n",
    "- Target variable that indicates if an ad was clicked (1) or not (0).\n",
    "\n",
    "#### I1-I13 \n",
    "- A total of 13 columns of integer features (mostly count features).\n",
    "\n",
    "#### C1-C26 \n",
    "- A total of 26 columns of categorical features. The values of these features have been hashed onto 32 bits for anonymization purposes. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据中含有Label字段，表示这条广告是否被点击，I1-I13一共13个数值特征（Dense Input），C1-C26共26个Categorical类别特征（Sparse Input）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 网络模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='model.png'\\>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "模型包含三部分网络，一个是FFM(Field-aware Factorization Machines)，一个是FM(Factorization Machine)，另一个是DNN，其中FM网络包含GBDT和FM两个组件。通常在数据预处理的部分，需要做特征交叉组合等特征工程，以便找出帮助我们预测的特征出来，这绝对是技术活。\n",
    "\n",
    "这次我们跳过特征工程的步骤，把这些组件和深度神经网络组合在一起，将挑选特征的工作交给模型来处理。其中FFM使用了[`LibFFM`](https://www.csie.ntu.edu.tw/~cjlin/libffm/)，FM使用了[`LibFM`](http://www.libfm.org)，GBDT使用了[`LightGBM`](https://github.com/Microsoft/LightGBM)，当然你也可以使用[`xgboost`](http://xgboost.readthedocs.io/en/latest/)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GBDT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "给入训练数据后，GBDT会训练出若干棵树，我们要使用的是GBDT中每棵树输出的叶子结点，将这些叶子结点作为categorical类别特征输入给FM。有关决策树的使用，请参照Facebook的这篇文章[`Practical Lessons from Predicting Clicks on Ads at Facebook`](http://quinonero.net/Publications/predicting-clicks-facebook.pdf)。\n",
    "<img src=\"facebook.png\"\\>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "FM用来解决数据量大并且特征稀疏下的特征组合问题，先来看看公式（只考虑二阶多项式的情况）：n代表样本的特征数量，$x_i$是第i个特征的值，$w_0$、$w_i$、$w_i$$_j$是模型参数。\n",
    "<img src=\"fm_formula.png\"\\>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从公式可以看出来这是在线性模型基础上，添加了特征组合$x_ix_j$，当然只有在特征$x_i$和$x_j$都不为0时才有意义。然而在实际的应用场景中，训练组合特征的参数是很困难的。因为输入数据普遍存在稀疏性，这导致$x_i$和$x_j$大部分情况都是0，而组合特征的参数$w_i$$_j$只有在特征不为0时才能训练出有意义的值。\n",
    "\n",
    "比如跟购物相关的特征中，女性可能会更关注化妆品或者首饰之类的物品，而男性可能更关注体育用品或者电子产品等商品，这说明特征组合训练是有意义的。而商品特征可能存在几百上千种分类，通常我们将类别特征转成One hot编码的形式，这样一个特征就要变成几百维的特征，再加上其他的分类特征，这导致输入的特征空间急剧膨胀，所以数据的稀疏性是实际问题中不可避免的挑战。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了解决二次项参数训练的问题，引入了矩阵分解的概念。在上一篇文章中我们讨论的是电影推荐系统，我们构造了用户特征向量和电影特征向量，通过两个特征向量的点积得到了用户对于某部电影的评分。如果将用户特征矩阵与电影特征矩阵相乘就会得到所有用户对所有影片的评分矩阵。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果将上面的过程反过来看，实际上对于评分矩阵，我们可以分解成用户矩阵和电影矩阵，而评分矩阵中每一个数据点就相当于上面讨论的组合特征的参数$w_i$$_j$。\n",
    "\n",
    "对于参数矩阵W，我们采用矩阵分解的方法，将每一个参数$w_i$$_j$分解成两个向量（称之为隐向量）的点积。这样矩阵就可以分解为$W=V^TV$，而每个参数$w_i$$_j$=⟨$v_i$,$v_j$⟩，$v_i$是第i维特征的隐向量，这样FM的二阶公式就变成：\n",
    "<img src=\"fm_formula2.png\"\\>\n",
    "\n",
    "这就是FM模型的思想。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将GBDT输出的叶子节点作为训练数据的输入，来训练FM模型。这样对于我们的FM网络，需要训练GBDT和FM。看得出来，这次我们的点击率预测网络要复杂了许多，影响最终结果的因素和超参更多了。关于FM和GBDT两个组件的训练我们会在下文进行说明。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FFM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来需要训练FFM模型。FFM在FM的基础上增加了一个Field的概念，比如说一个商品字段，是一个分类特征，可以分成很多不同的feature，但是这些feature都属于同一个Field，或者说同一个categorical的分类特征都可以放到同一个Field。\n",
    "\n",
    "这可以看成是1对多的关系，打个比方，比如职业字段，这是一个特征，经过One Hot以后，变成了N个特征。那这N个特征其实都属于职业，所以职业就是一个Field。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们要通过特征组合来训练隐向量，这样每一维特征$x_i$，都会与其他特征的每一种Field $f_j$学习一个隐向量$v_{i,f_j}$。也就是说，隐向量不仅与特征有关，还与Field有关。模型的公式：\n",
    "<img src=\"ffm_formula.png\"\\>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们来看DNN的部分。将输入数据分成两部分，一部分是数值特征（Dense Input），一部分是类别特征（Sparse Input）。我们仍然不适用One Hot编码，将类别特征传入嵌入层，得到多个嵌入向量，再将这些嵌入向量和数值特征连接在一起，传入全连接层，一共连接三层全连接层，使用Relu激活函数。然后再将第三层全连接的输出和FFM、FM的全连接层的输出连接在一起，传入最后一层全连接层。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们要学习的目标Label表示广告是否被点击了，只有1（点击）和0（没有点击）两种状态。所以我们网络的最后一层要做Logistic回归，在最后一层全连接层使用Sigmoid激活函数，得到广告被点击的概率。\n",
    "\n",
    "使用LogLoss作为损失函数，FTRL作为学习算法。\n",
    "\n",
    "FTRL有关的Paper：[`Ad_click_prediction_a_view_from_the_trenches`](https://www.researchgate.net/publication/262412214_Ad_click_prediction_a_view_from_the_trenches)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### LibFFM和LibFM的代码我做了修改，请使用代码库中我的相关代码。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 预处理数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 生成神经网络的输入\n",
    "- 生成FFM的输入\n",
    "- 生成GBDT的输入"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "首先要为DNN、FFM和GBDT的输入做预处理。对于数值特征，我们将I1-I13转成0-1之间的小数。类别特征我们将某类别使用次数少于cutoff（超参）的忽略掉，留下使用次数多的feature作为某类别字段的特征，然后将这些特征以各自字段为组进行编号。\n",
    "\n",
    "比如有C1和C2两个类别字段，C1下面有特征a（大于cutoff次）、b（少于cutoff次）、c（大于cutoff次），C2下面有特征x和y（均大于cutoff次），这样留下来的特征就是C1：a、c和C2：x、y。然后以各自字段为分组进行编号，对于C1字段，a和c的特征id对应0和1；对于C2字段，x和y也是0和1。\n",
    "\n",
    "对于类别特征的输入数据处理，FFM和GBDT各不相同，我们分别来说。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GBDT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "GBDT的处理要简单一些，C1-C26每个字段各自的特征id值作为输入即可。\n",
    "GBDT的输入数据格式是：Label I1-I13 C1-C26\n",
    "所以实际输入可能是这样：0 小数1 小数2 ~ 小数13 1（C1特征Id） 0（C2特征Id） ~ C26特征Id\n",
    "其中C1特征Id是1，说明此处C1字段的feature是c，而C2字段的feature是x。\n",
    "\n",
    "下面是一段生成的真实数据：\n",
    "0\t0.05\t0.004983\t0.05\t0\t0.021594\t0.008\t0.15\t0.04\t0.362\t0.166667\t0.2\t0\t0.04\t2\t3\t0\t0\t1\t1\t0\t3\t1\t0\t0\t0\t0\t3\t0\t0\t1\t4\t1\t3\t0\t0\t2\t0\t1\t0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "很抱歉，我的造句能力实在很差，要是上面一段文字看的你很混乱的话，那就直接看代码吧：）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FFM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "FFM的输入数据要复杂一些，详细可以参看官方[`Github`](https://github.com/guestwalk/libffm)上的说明，摘抄如下：\n",
    "\n",
    "It is important to understand the difference between `field` and `feature`. For example, if we have a raw data like this:\n",
    "\n",
    "    Click  Advertiser  Publisher\n",
    "    =====  ==========  =========\n",
    "    0        Nike        CNN\n",
    "    1        ESPN        BBC\n",
    "\n",
    "Here, we have \n",
    "\n",
    "    * 2 fields: Advertiser and Publisher\n",
    "    * 4 features: Advertiser-Nike, Advertiser-ESPN, Publisher-CNN, Publisher-BBC\n",
    "\n",
    "Usually you will need to build two dictionares, one for field and one for features, like this:\n",
    "    \n",
    "    DictField[Advertiser] -> 0\n",
    "    DictField[Publisher]  -> 1\n",
    "    \n",
    "    DictFeature[Advertiser-Nike] -> 0\n",
    "    DictFeature[Publisher-CNN]   -> 1\n",
    "    DictFeature[Advertiser-ESPN] -> 2\n",
    "    DictFeature[Publisher-BBC]   -> 3\n",
    "\n",
    "Then, you can generate FFM format data:\n",
    "\n",
    "    0 0:0:1 1:1:1\n",
    "    1 0:2:1 1:3:1\n",
    "\n",
    "Note that because these features are categorical, the values here are all ones.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "fields应该很好理解，features的划分跟之前GBDT有些不一样，在刚刚GBDT的处理中我们是每个类别内独立编号，C1有features 0~n，C2有features 0~n。而这次FFM是所有的features统一起来编号。你看它的例子，C1是Advertiser，有两个feature，C2是Publisher，有两个feature，统一起来编号就是0~3。而在GBDT我们要独立编号的，看起来像这样：\n",
    "\n",
    "    DictFeature[Advertiser-Nike] -> 0\n",
    "    DictFeature[Advertiser-ESPN] -> 1\n",
    "    DictFeature[Publisher-CNN]   -> 0\n",
    "    DictFeature[Publisher-BBC]   -> 1 \n",
    "现在我们假设有第三条数据，看看如何构造FFM的输入数据：\n",
    "\n",
    "    Click  Advertiser  Publisher\n",
    "    =====  ==========  =========\n",
    "    0        Nike        CNN\n",
    "    1        ESPN        BBC\n",
    "    0        Lining      CNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "按照规则，应该是像下面这样：\n",
    "\n",
    "    DictFeature[Advertiser-Nike]   -> 0\n",
    "    DictFeature[Publisher-CNN]     -> 1\n",
    "    DictFeature[Advertiser-ESPN]   -> 2\n",
    "    DictFeature[Publisher-BBC]     -> 3\n",
    "    DictFeature[Advertiser-Lining] -> 4\n",
    "在我们这次FFM的输入数据处理中，跟上面略有些区别，每个类别编号以后，下一个类别继续编号，所以最终的features编号是这样的：\n",
    "\n",
    "    DictFeature[Advertiser-Nike]   -> 0\n",
    "    DictFeature[Advertiser-ESPN]   -> 1\n",
    "    DictFeature[Advertiser-Lining] -> 2\n",
    "    DictFeature[Publisher-CNN]     -> 3\n",
    "    DictFeature[Publisher-BBC]     -> 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于我们的数据是从I1开始编号的，从I1-I13，所以C1的编号要从加13开始。\n",
    "\n",
    "这是一条来自真实的FFM输入数据：\n",
    "0\t0:0:0.05\t1:1:0.004983\t2:2:0.05\t3:3:0\t4:4:0.021594\t5:5:0.008\t6:6:0.15\t7:7:0.04\t8:8:0.362\t9:9:0.166667\t10:10:0.2\t11:11:0\t12:12:0.04\t13:15:1\t14:29:1\t15:64:1\t16:76:1\t17:92:1\t18:101:1\t19:107:1\t20:122:1\t21:131:1\t22:133:1\t23:143:1\t24:166:1\t25:179:1\t26:209:1\t27:216:1\t28:243:1\t29:260:1\t30:273:1\t31:310:1\t32:317:1\t33:318:1\t34:333:1\t35:340:1\t36:348:1\t37:368:1\t38:381:1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "DNN的输入数据就没有那么复杂了，仍然是I1-I13的小数和C1-C26的统一编号，就像FFM一样，只是不需要从加13开始，最后是Label。\n",
    "真实数据就像这样：\n",
    "0.05,0.004983,0.05,0,0.021594,0.008,0.15,0.04,0.362,0.166667,0.2,0,0.04,2,16,51,63,79,88,94,109,118,120,130,153,166,196,203,230,247,260,297,304,305,320,\n",
    "327,335,355,368,0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "要说明的就这么多了，我们来看看代码吧，因为要同时生成训练数据、验证数据和测试数据，所以要运行一段时间。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "import click\n",
    "import random\n",
    "import collections\n",
    "\n",
    "import numpy as np\n",
    "import lightgbm as lgb\n",
    "import json\n",
    "import lightgbm as lgb\n",
    "import pandas as pd\n",
    "from sklearn.metrics import mean_squared_error\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "\n",
    "def save_params(params):\n",
    "    \"\"\"\n",
    "    Save parameters to file\n",
    "    \"\"\"\n",
    "    pickle.dump(params, open('params.p', 'wb'))\n",
    "\n",
    "\n",
    "def load_params():\n",
    "    \"\"\"\n",
    "    Load parameters from file\n",
    "    \"\"\"\n",
    "    return pickle.load(open('params.p', mode='rb'))\n",
    "\n",
    "\n",
    "def save_params_with_name(params, name):\n",
    "    \"\"\"\n",
    "    Save parameters to file\n",
    "    \"\"\"\n",
    "    pickle.dump(params, open('{}.p'.format(name), 'wb'))\n",
    "\n",
    "\n",
    "def load_params_with_name(name):\n",
    "    \"\"\"\n",
    "    Load parameters from file\n",
    "    \"\"\"\n",
    "    return pickle.load(open('{}.p'.format(name), mode='rb'))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下代码来自百度deep_fm的[`preprocess.py`](https://github.com/PaddlePaddle/models/blob/develop/deep_fm/preprocess.py)，稍稍添了些代码，我就不重复造轮子了：）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# There are 13 integer features and 26 categorical features\n",
    "continous_features = range(1, 14)\n",
    "categorial_features = range(14, 40)\n",
    "\n",
    "# Clip integer features. The clip point for each integer feature\n",
    "# is derived from the 95% quantile of the total values in each feature\n",
    "continous_clip = [20, 600, 100, 50, 64000, 500, 100, 50, 500, 10, 10, 10, 50]\n",
    "\n",
    "class ContinuousFeatureGenerator:\n",
    "    \"\"\"\n",
    "    Normalize the integer features to [0, 1] by min-max normalization\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, num_feature):\n",
    "        self.num_feature = num_feature\n",
    "        self.min = [sys.maxsize] * num_feature\n",
    "        self.max = [-sys.maxsize] * num_feature\n",
    "\n",
    "    def build(self, datafile, continous_features):\n",
    "        with open(datafile, 'r') as f:\n",
    "            for line in f:\n",
    "                features = line.rstrip('\\n').split('\\t')\n",
    "                for i in range(0, self.num_feature):\n",
    "                    val = features[continous_features[i]]\n",
    "                    if val != '':\n",
    "                        val = int(val)\n",
    "                        if val > continous_clip[i]:\n",
    "                            val = continous_clip[i]\n",
    "                        self.min[i] = min(self.min[i], val)\n",
    "                        self.max[i] = max(self.max[i], val)\n",
    "\n",
    "    def gen(self, idx, val):\n",
    "        if val == '':\n",
    "            return 0.0\n",
    "        val = float(val)\n",
    "        return (val - self.min[idx]) / (self.max[idx] - self.min[idx])\n",
    "\n",
    "class CategoryDictGenerator:\n",
    "    \"\"\"\n",
    "    Generate dictionary for each of the categorical features\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, num_feature):\n",
    "        self.dicts = []\n",
    "        self.num_feature = num_feature\n",
    "        for i in range(0, num_feature):\n",
    "            self.dicts.append(collections.defaultdict(int))\n",
    "\n",
    "    def build(self, datafile, categorial_features, cutoff=0):\n",
    "        with open(datafile, 'r') as f:\n",
    "            for line in f:\n",
    "                features = line.rstrip('\\n').split('\\t')\n",
    "                for i in range(0, self.num_feature):\n",
    "                    if features[categorial_features[i]] != '':\n",
    "                        self.dicts[i][features[categorial_features[i]]] += 1\n",
    "        for i in range(0, self.num_feature):\n",
    "            self.dicts[i] = filter(lambda x: x[1] >= cutoff,\n",
    "                                   self.dicts[i].items())\n",
    "\n",
    "            self.dicts[i] = sorted(self.dicts[i], key=lambda x: (-x[1], x[0]))\n",
    "            vocabs, _ = list(zip(*self.dicts[i]))\n",
    "            self.dicts[i] = dict(zip(vocabs, range(1, len(vocabs) + 1)))\n",
    "            self.dicts[i]['<unk>'] = 0\n",
    "\n",
    "    def gen(self, idx, key):\n",
    "        if key not in self.dicts[idx]:\n",
    "            res = self.dicts[idx]['<unk>']\n",
    "        else:\n",
    "            res = self.dicts[idx][key]\n",
    "        return res\n",
    "\n",
    "    def dicts_sizes(self):\n",
    "        return list(map(len, self.dicts))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def preprocess(datadir, outdir):\n",
    "    \"\"\"\n",
    "    All the 13 integer features are normalzied to continous values and these\n",
    "    continous features are combined into one vecotr with dimension 13.\n",
    "\n",
    "    Each of the 26 categorical features are one-hot encoded and all the one-hot\n",
    "    vectors are combined into one sparse binary vector.\n",
    "    \"\"\"\n",
    "    dists = ContinuousFeatureGenerator(len(continous_features))\n",
    "    dists.build(os.path.join(datadir, 'train.txt'), continous_features)\n",
    "\n",
    "    dicts = CategoryDictGenerator(len(categorial_features))\n",
    "    dicts.build(\n",
    "        os.path.join(datadir, 'train.txt'), categorial_features, cutoff=200)#200 50\n",
    "\n",
    "    dict_sizes = dicts.dicts_sizes()\n",
    "    categorial_feature_offset = [0]\n",
    "    for i in range(1, len(categorial_features)):\n",
    "        offset = categorial_feature_offset[i - 1] + dict_sizes[i - 1]\n",
    "        categorial_feature_offset.append(offset)\n",
    "\n",
    "    random.seed(0)\n",
    "\n",
    "    # 90% of the data are used for training, and 10% of the data are used\n",
    "    # for validation.\n",
    "    train_ffm = open(os.path.join(outdir, 'train_ffm.txt'), 'w')\n",
    "    valid_ffm = open(os.path.join(outdir, 'valid_ffm.txt'), 'w')\n",
    "\n",
    "    train_lgb = open(os.path.join(outdir, 'train_lgb.txt'), 'w')\n",
    "    valid_lgb = open(os.path.join(outdir, 'valid_lgb.txt'), 'w')\n",
    "\n",
    "    with open(os.path.join(outdir, 'train.txt'), 'w') as out_train:\n",
    "        with open(os.path.join(outdir, 'valid.txt'), 'w') as out_valid:\n",
    "            with open(os.path.join(datadir, 'train.txt'), 'r') as f:\n",
    "                for line in f:\n",
    "                    features = line.rstrip('\\n').split('\\t')\n",
    "                    continous_feats = []\n",
    "                    continous_vals = []\n",
    "                    for i in range(0, len(continous_features)):\n",
    "\n",
    "                        val = dists.gen(i, features[continous_features[i]])\n",
    "                        continous_vals.append(\n",
    "                            \"{0:.6f}\".format(val).rstrip('0').rstrip('.'))\n",
    "                        continous_feats.append(\n",
    "                            \"{0:.6f}\".format(val).rstrip('0').rstrip('.'))#('{0}'.format(val))\n",
    "\n",
    "                    categorial_vals = []\n",
    "                    categorial_lgb_vals = []\n",
    "                    for i in range(0, len(categorial_features)):\n",
    "                        val = dicts.gen(i, features[categorial_features[i]]) + categorial_feature_offset[i]\n",
    "                        categorial_vals.append(str(val))\n",
    "                        val_lgb = dicts.gen(i, features[categorial_features[i]])\n",
    "                        categorial_lgb_vals.append(str(val_lgb))\n",
    "\n",
    "                    continous_vals = ','.join(continous_vals)\n",
    "                    categorial_vals = ','.join(categorial_vals)\n",
    "                    label = features[0]\n",
    "                    if random.randint(0, 9999) % 10 != 0:\n",
    "                        out_train.write(','.join(\n",
    "                            [continous_vals, categorial_vals, label]) + '\\n')\n",
    "                        train_ffm.write('\\t'.join(label) + '\\t')\n",
    "                        train_ffm.write('\\t'.join(\n",
    "                            ['{}:{}:{}'.format(ii, ii, val) for ii,val in enumerate(continous_vals.split(','))]) + '\\t')\n",
    "                        train_ffm.write('\\t'.join(\n",
    "                            ['{}:{}:1'.format(ii + 13, str(np.int32(val) + 13)) for ii, val in enumerate(categorial_vals.split(','))]) + '\\n')\n",
    "                        \n",
    "                        train_lgb.write('\\t'.join(label) + '\\t')\n",
    "                        train_lgb.write('\\t'.join(continous_feats) + '\\t')\n",
    "                        train_lgb.write('\\t'.join(categorial_lgb_vals) + '\\n')\n",
    "\n",
    "                    else:\n",
    "                        out_valid.write(','.join(\n",
    "                            [continous_vals, categorial_vals, label]) + '\\n')\n",
    "                        valid_ffm.write('\\t'.join(label) + '\\t')\n",
    "                        valid_ffm.write('\\t'.join(\n",
    "                            ['{}:{}:{}'.format(ii, ii, val) for ii,val in enumerate(continous_vals.split(','))]) + '\\t')\n",
    "                        valid_ffm.write('\\t'.join(\n",
    "                            ['{}:{}:1'.format(ii + 13, str(np.int32(val) + 13)) for ii, val in enumerate(categorial_vals.split(','))]) + '\\n')\n",
    "                                                \n",
    "                        valid_lgb.write('\\t'.join(label) + '\\t')\n",
    "                        valid_lgb.write('\\t'.join(continous_feats) + '\\t')\n",
    "                        valid_lgb.write('\\t'.join(categorial_lgb_vals) + '\\n')\n",
    "                        \n",
    "    train_ffm.close()\n",
    "    valid_ffm.close()\n",
    "\n",
    "    train_lgb.close()\n",
    "    valid_lgb.close()\n",
    "\n",
    "    test_ffm = open(os.path.join(outdir, 'test_ffm.txt'), 'w')\n",
    "    test_lgb = open(os.path.join(outdir, 'test_lgb.txt'), 'w')\n",
    "\n",
    "    with open(os.path.join(outdir, 'test.txt'), 'w') as out:\n",
    "        with open(os.path.join(datadir, 'test.txt'), 'r') as f:\n",
    "            for line in f:\n",
    "                features = line.rstrip('\\n').split('\\t')\n",
    "\n",
    "                continous_feats = []\n",
    "                continous_vals = []\n",
    "                for i in range(0, len(continous_features)):\n",
    "                    val = dists.gen(i, features[continous_features[i] - 1])\n",
    "                    continous_vals.append(\n",
    "                        \"{0:.6f}\".format(val).rstrip('0').rstrip('.'))\n",
    "                    continous_feats.append(\n",
    "                            \"{0:.6f}\".format(val).rstrip('0').rstrip('.'))#('{0}'.format(val))\n",
    "\n",
    "                categorial_vals = []\n",
    "                categorial_lgb_vals = []\n",
    "                for i in range(0, len(categorial_features)):\n",
    "                    val = dicts.gen(i,\n",
    "                                    features[categorial_features[i] -\n",
    "                                             1]) + categorial_feature_offset[i]\n",
    "                    categorial_vals.append(str(val))\n",
    "\n",
    "                    val_lgb = dicts.gen(i, features[categorial_features[i] - 1])\n",
    "                    categorial_lgb_vals.append(str(val_lgb))\n",
    "\n",
    "                continous_vals = ','.join(continous_vals)\n",
    "                categorial_vals = ','.join(categorial_vals)\n",
    "\n",
    "                out.write(','.join([continous_vals, categorial_vals]) + '\\n')\n",
    "                \n",
    "                test_ffm.write('\\t'.join(['{}:{}:{}'.format(ii, ii, val) for ii,val in enumerate(continous_vals.split(','))]) + '\\t')\n",
    "                test_ffm.write('\\t'.join(\n",
    "                    ['{}:{}:1'.format(ii + 13, str(np.int32(val) + 13)) for ii, val in enumerate(categorial_vals.split(','))]) + '\\n')\n",
    "                                                                \n",
    "                test_lgb.write('\\t'.join(continous_feats) + '\\t')\n",
    "                test_lgb.write('\\t'.join(categorial_lgb_vals) + '\\n')\n",
    "\n",
    "    test_ffm.close()\n",
    "    test_lgb.close()\n",
    "    return dict_sizes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "dict_sizes = preprocess('./data/raw','./data')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_params_with_name((dict_sizes), 'dict_sizes') #pickle.dump((dict_sizes), open('dict_sizes.p', 'wb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dict_sizes = load_params_with_name('dict_sizes') #pickle.load(open('dict_sizes.p', mode='rb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8496"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(dict_sizes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练FFM\n",
    "\n",
    "数据准备好了，开始调用LibFFM，训练FFM模型。\n",
    "\n",
    "learning rate是0.1，迭代32次，训练好后保存的模型文件是model_ffm。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import subprocess, sys, os, time\n",
    "\n",
    "NR_THREAD = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['First check if the text file has already been converted to binary format (1.3 seconds)\\n',\n",
       " 'Binary file found. Skip converting text to binary\\n',\n",
       " 'First check if the text file has already been converted to binary format (0.2 seconds)\\n',\n",
       " 'Binary file found. Skip converting text to binary\\n',\n",
       " 'iter   tr_logloss   va_logloss      tr_time\\n',\n",
       " '   1      0.49339      0.48196         12.8\\n',\n",
       " '   2      0.47621      0.47651         25.9\\n',\n",
       " '   3      0.47149      0.47433         39.0\\n',\n",
       " '   4      0.46858      0.47277         51.2\\n',\n",
       " '   5      0.46630      0.47168         63.0\\n',\n",
       " '   6      0.46447      0.47092         74.7\\n',\n",
       " '   7      0.46269      0.47038         86.4\\n',\n",
       " '   8      0.46113      0.47000         98.0\\n',\n",
       " '   9      0.45960      0.46960        109.6\\n',\n",
       " '  10      0.45811      0.46940        121.2\\n',\n",
       " '  11      0.45660      0.46913        132.5\\n',\n",
       " '  12      0.45509      0.46899        144.3\\n',\n",
       " '  13      0.45366      0.46903\\n',\n",
       " 'Auto-stop. Use model at 12th iteration.\\n']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cmd = './libffm/libffm/ffm-train --auto-stop -r 0.1 -t 32 -s {nr_thread} -p ./data/valid_ffm.txt ./data/train_ffm.txt model_ffm'.format(nr_thread=NR_THREAD) \n",
    "os.popen(cmd).readlines()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "FFM模型训练好了，我们把训练、验证和测试数据输入给FFM，得到FFM层的输出，输出的文件名为*.out.logit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['logloss = 0.45308\\n']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cmd = './libffm/libffm/ffm-predict ./data/train_ffm.txt model_ffm tr_ffm.out'.format(nr_thread=NR_THREAD) \n",
    "os.popen(cmd).readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['logloss = 0.46899\\n']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cmd = './libffm/libffm/ffm-predict ./data/valid_ffm.txt model_ffm va_ffm.out'.format(nr_thread=NR_THREAD) \n",
    "os.popen(cmd).readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['done!\\n']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cmd = './libffm/libffm/ffm-predict ./data/test_ffm.txt model_ffm te_ffm.out true'.format(nr_thread=NR_THREAD) \n",
    "os.popen(cmd).readlines()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练GBDT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在调用LightGBM训练GBDT模型，因为决策树较容易过拟合，我们设置树的个数为32，叶子节点数设为30，深度就不设置了，学习率设为0.05。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def lgb_pred(tr_path, va_path, _sep = '\\t', iter_num = 32):\n",
    "    # load or create your dataset\n",
    "    print('Load data...')\n",
    "    df_train = pd.read_csv(tr_path, header=None, sep=_sep)\n",
    "    df_test = pd.read_csv(va_path, header=None, sep=_sep)\n",
    "    \n",
    "    y_train = df_train[0].values\n",
    "    y_test = df_test[0].values\n",
    "    X_train = df_train.drop(0, axis=1).values\n",
    "    X_test = df_test.drop(0, axis=1).values\n",
    "    \n",
    "    # create dataset for lightgbm\n",
    "    lgb_train = lgb.Dataset(X_train, y_train)\n",
    "    lgb_eval = lgb.Dataset(X_test, y_test, reference=lgb_train)\n",
    "    \n",
    "    # specify your configurations as a dict\n",
    "    params = {\n",
    "        'task': 'train',\n",
    "        'boosting_type': 'gbdt',\n",
    "        'objective': 'binary',\n",
    "        'metric': {'l2', 'auc', 'logloss'},\n",
    "        'num_leaves': 30,\n",
    "#         'max_depth': 7,\n",
    "        'num_trees': 32,\n",
    "        'learning_rate': 0.05,\n",
    "        'feature_fraction': 0.9,\n",
    "        'bagging_fraction': 0.8,\n",
    "        'bagging_freq': 5,\n",
    "        'verbose': 0\n",
    "    }\n",
    "    \n",
    "    print('Start training...')\n",
    "    # train\n",
    "    gbm = lgb.train(params,\n",
    "                    lgb_train,\n",
    "                    num_boost_round=iter_num,\n",
    "                    valid_sets=lgb_eval,\n",
    "                    feature_name=[\"I1\",\"I2\",\"I3\",\"I4\",\"I5\",\"I6\",\"I7\",\"I8\",\"I9\",\"I10\",\"I11\",\"I12\",\"I13\",\"C1\",\"C2\",\"C3\",\"C4\",\"C5\",\"C6\",\"C7\",\"C8\",\"C9\",\"C10\",\"C11\",\"C12\",\"C13\",\"C14\",\"C15\",\"C16\",\"C17\",\"C18\",\"C19\",\"C20\",\"C21\",\"C22\",\"C23\",\"C24\",\"C25\",\"C26\"],\n",
    "                    categorical_feature=[\"C1\",\"C2\",\"C3\",\"C4\",\"C5\",\"C6\",\"C7\",\"C8\",\"C9\",\"C10\",\"C11\",\"C12\",\"C13\",\"C14\",\"C15\",\"C16\",\"C17\",\"C18\",\"C19\",\"C20\",\"C21\",\"C22\",\"C23\",\"C24\",\"C25\",\"C26\"],\n",
    "                    early_stopping_rounds=5)\n",
    "    \n",
    "    print('Save model...')\n",
    "    # save model to file\n",
    "    gbm.save_model('lgb_model.txt')\n",
    "    \n",
    "    print('Start predicting...')\n",
    "    # predict\n",
    "    y_pred = gbm.predict(X_test, num_iteration=gbm.best_iteration)\n",
    "    # eval\n",
    "    print('The rmse of prediction is:', mean_squared_error(y_test, y_pred) ** 0.5)\n",
    "\n",
    "    return gbm,y_pred,X_train,y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Load data...\n",
      "Start training...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda/envs/tensorflow1.0/lib/python3.5/site-packages/lightgbm/engine.py:99: UserWarning: Found `num_trees` in params. Will use it instead of argument\n",
      "  warnings.warn(\"Found `{}` in params. Will use it instead of argument\".format(alias))\n",
      "/Applications/anaconda/envs/tensorflow1.0/lib/python3.5/site-packages/lightgbm/basic.py:1029: UserWarning: categorical_feature in Dataset is overrided. New categorical_feature is ['C1', 'C10', 'C11', 'C12', 'C13', 'C14', 'C15', 'C16', 'C17', 'C18', 'C19', 'C2', 'C20', 'C21', 'C22', 'C23', 'C24', 'C25', 'C26', 'C3', 'C4', 'C5', 'C6', 'C7', 'C8', 'C9']\n",
      "  warnings.warn('categorical_feature in Dataset is overrided. New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n",
      "/Applications/anaconda/envs/tensorflow1.0/lib/python3.5/site-packages/lightgbm/basic.py:668: UserWarning: categorical_feature in param dict is overrided.\n",
      "  warnings.warn('categorical_feature in param dict is overrided.')\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1]\tvalid_0's l2: 0.241954\tvalid_0's auc: 0.70607\n",
      "Training until validation scores don't improve for 5 rounds.\n",
      "[2]\tvalid_0's l2: 0.234704\tvalid_0's auc: 0.715608\n",
      "[3]\tvalid_0's l2: 0.228139\tvalid_0's auc: 0.717791\n",
      "[4]\tvalid_0's l2: 0.222168\tvalid_0's auc: 0.72273\n",
      "[5]\tvalid_0's l2: 0.216728\tvalid_0's auc: 0.724065\n",
      "[6]\tvalid_0's l2: 0.211819\tvalid_0's auc: 0.725036\n",
      "[7]\tvalid_0's l2: 0.207316\tvalid_0's auc: 0.727427\n",
      "[8]\tvalid_0's l2: 0.203296\tvalid_0's auc: 0.728583\n",
      "[9]\tvalid_0's l2: 0.199582\tvalid_0's auc: 0.730092\n",
      "[10]\tvalid_0's l2: 0.196185\tvalid_0's auc: 0.730792\n",
      "[11]\tvalid_0's l2: 0.193063\tvalid_0's auc: 0.732316\n",
      "[12]\tvalid_0's l2: 0.190268\tvalid_0's auc: 0.733773\n",
      "[13]\tvalid_0's l2: 0.187697\tvalid_0's auc: 0.734782\n",
      "[14]\tvalid_0's l2: 0.185351\tvalid_0's auc: 0.735636\n",
      "[15]\tvalid_0's l2: 0.183215\tvalid_0's auc: 0.736346\n",
      "[16]\tvalid_0's l2: 0.181241\tvalid_0's auc: 0.737393\n",
      "[17]\tvalid_0's l2: 0.179468\tvalid_0's auc: 0.737709\n",
      "[18]\tvalid_0's l2: 0.177829\tvalid_0's auc: 0.739096\n",
      "[19]\tvalid_0's l2: 0.176326\tvalid_0's auc: 0.740135\n",
      "[20]\tvalid_0's l2: 0.174948\tvalid_0's auc: 0.741065\n",
      "[21]\tvalid_0's l2: 0.173675\tvalid_0's auc: 0.742165\n",
      "[22]\tvalid_0's l2: 0.172499\tvalid_0's auc: 0.742672\n",
      "[23]\tvalid_0's l2: 0.171471\tvalid_0's auc: 0.743246\n",
      "[24]\tvalid_0's l2: 0.17045\tvalid_0's auc: 0.744415\n",
      "[25]\tvalid_0's l2: 0.169582\tvalid_0's auc: 0.744792\n",
      "[26]\tvalid_0's l2: 0.168746\tvalid_0's auc: 0.745478\n",
      "[27]\tvalid_0's l2: 0.167966\tvalid_0's auc: 0.746282\n",
      "[28]\tvalid_0's l2: 0.167264\tvalid_0's auc: 0.74675\n",
      "[29]\tvalid_0's l2: 0.166582\tvalid_0's auc: 0.747429\n",
      "[30]\tvalid_0's l2: 0.16594\tvalid_0's auc: 0.748392\n",
      "[31]\tvalid_0's l2: 0.165364\tvalid_0's auc: 0.748986\n",
      "[32]\tvalid_0's l2: 0.164844\tvalid_0's auc: 0.749362\n",
      "Did not meet early stopping. Best iteration is:\n",
      "[32]\tvalid_0's l2: 0.164844\tvalid_0's auc: 0.749362\n",
      "Save model...\n",
      "Start predicting...\n",
      "The rmse of prediction is: 0.406009502303\n"
     ]
    }
   ],
   "source": [
    "gbm,y_pred,X_train ,y_train = lgb_pred('./data/train_lgb.txt', './data/valid_lgb.txt', '\\t', 256)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 查看每个特征的重要程度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 15,   0,  30,  10,  12,  79,  31,  15,  14,   0,  44,   0,  29,\n",
       "         0,  16,   0,  65,   0,   0,  32,   0,   0,  29,  30,  10, 120,\n",
       "        30, 165,  19,  11,  69,   1,   0,   2,   0,   6,  39,   0,   5])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gbm.feature_importance()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  69634.31561279,       0.        ,   17624.44689941,\n",
       "          4734.61398315,   10529.7180481 ,  199794.76257324,\n",
       "         94191.14331055,   13543.23699951,   10014.74700928,\n",
       "             0.        ,  191050.53414917,       0.        ,\n",
       "         28020.85171509,       0.        ,    6852.7729187 ,\n",
       "             0.        ,   32251.70903015,       0.        ,\n",
       "             0.        ,   14341.38494873,       0.        ,\n",
       "             0.        ,   11129.02203369,   12486.21105957,\n",
       "          5218.96902466,   99722.85806274,   23106.2180481 ,\n",
       "         79130.2718811 ,   10490.07904053,   17757.50100708,\n",
       "         34302.44396973,     424.67401123,       0.        ,\n",
       "           882.20599365,       0.        ,    3156.61196899,\n",
       "         15901.01004028,       0.        ,    3397.2270813 ])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gbm.feature_importance(\"gain\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 我们把每个特征的重要程度排个序看看"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ret_feat_impt(gbm):\n",
    "    gain = gbm.feature_importance(\"gain\").reshape(-1, 1) / sum(gbm.feature_importance(\"gain\"))\n",
    "    col = np.array(gbm.feature_name()).reshape(-1, 1)\n",
    "    return sorted(np.column_stack((col, gain)),key=lambda x: x[1],reverse=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array(['I6', '0.1978774213012332'],\n",
       "       dtype='<U32'), array(['I11', '0.1892171073393491'],\n",
       "       dtype='<U32'), array(['C13', '0.09876586224832032'],\n",
       "       dtype='<U32'), array(['I7', '0.09328723289667494'],\n",
       "       dtype='<U32'), array(['C15', '0.07837089393651243'],\n",
       "       dtype='<U32'), array(['I1', '0.06896606612740637'],\n",
       "       dtype='<U32'), array(['C18', '0.03397325870627491'],\n",
       "       dtype='<U32'), array(['C4', '0.03194220375573926'],\n",
       "       dtype='<U32'), array(['I13', '0.027751948092299045'],\n",
       "       dtype='<U32'), array(['C14', '0.022884477973766117'],\n",
       "       dtype='<U32'), array(['C17', '0.01758709018584479'],\n",
       "       dtype='<U32'), array(['I3', '0.01745531293913725'],\n",
       "       dtype='<U32'), array(['C24', '0.015748415135270675'],\n",
       "       dtype='<U32'), array(['C7', '0.014203757070472703'],\n",
       "       dtype='<U32'), array(['I8', '0.013413268591324624'],\n",
       "       dtype='<U32'), array(['C11', '0.012366386458128355'],\n",
       "       dtype='<U32'), array(['C10', '0.011022221770323784'],\n",
       "       dtype='<U32'), array(['I5', '0.01042866903792042'],\n",
       "       dtype='<U32'), array(['C16', '0.010389410428237439'],\n",
       "       dtype='<U32'), array(['I9', '0.009918639946598076'],\n",
       "       dtype='<U32'), array(['C2', '0.006787009911825981'],\n",
       "       dtype='<U32'), array(['C12', '0.005168884905437884'],\n",
       "       dtype='<U32'), array(['I4', '0.00468917800335175'],\n",
       "       dtype='<U32'), array(['C26', '0.003364625407413743'],\n",
       "       dtype='<U32'), array(['C23', '0.0031263193710805628'],\n",
       "       dtype='<U32'), array(['C21', '0.0008737398560005959'],\n",
       "       dtype='<U32'), array(['C19', '0.00042059860405565207'],\n",
       "       dtype='<U32'), array(['I2', '0.0'],\n",
       "       dtype='<U32'), array(['I10', '0.0'],\n",
       "       dtype='<U32'), array(['I12', '0.0'],\n",
       "       dtype='<U32'), array(['C1', '0.0'],\n",
       "       dtype='<U32'), array(['C3', '0.0'],\n",
       "       dtype='<U32'), array(['C5', '0.0'],\n",
       "       dtype='<U32'), array(['C6', '0.0'],\n",
       "       dtype='<U32'), array(['C8', '0.0'],\n",
       "       dtype='<U32'), array(['C9', '0.0'],\n",
       "       dtype='<U32'), array(['C20', '0.0'],\n",
       "       dtype='<U32'), array(['C22', '0.0'],\n",
       "       dtype='<U32'), array(['C25', '0.0'],\n",
       "       dtype='<U32')]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ret_feat_impt(gbm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 保存GBDT参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "dump = gbm.dump_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_params_with_name((gbm, dump), 'gbm_dump') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "gbm, dump = load_params_with_name('gbm_dump') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 通过eli5分析参数\n",
    "超级慢，看看就好，谨慎运行代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "import eli5 \n",
    "\n",
    "from sklearn.feature_extraction import DictVectorizer\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "from sklearn.utils import shuffle\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'C1': '2',\n",
       "  'C10': '7450',\n",
       "  'C11': '11013',\n",
       "  'C12': '11623',\n",
       "  'C13': '14481',\n",
       "  'C14': '14962',\n",
       "  'C15': '15302',\n",
       "  'C16': '16935',\n",
       "  'C17': '18436',\n",
       "  'C18': '18472',\n",
       "  'C19': '19603',\n",
       "  'C2': '152',\n",
       "  'C20': '20105',\n",
       "  'C21': '20107',\n",
       "  'C22': '21538',\n",
       "  'C23': '21549',\n",
       "  'C24': '22706',\n",
       "  'C25': '23114',\n",
       "  'C26': '24074',\n",
       "  'C3': '556',\n",
       "  'C4': '1896',\n",
       "  'C5': '3633',\n",
       "  'C6': '3684',\n",
       "  'C7': '6847',\n",
       "  'C8': '6887',\n",
       "  'C9': '6966',\n",
       "  'I1': '0.05',\n",
       "  'I10': '0.125',\n",
       "  'I11': '0.2',\n",
       "  'I12': '0',\n",
       "  'I13': '0.04',\n",
       "  'I2': '0.004983',\n",
       "  'I3': '0.05',\n",
       "  'I4': '0',\n",
       "  'I5': '0.021594',\n",
       "  'I6': '0.008',\n",
       "  'I7': '0.15',\n",
       "  'I8': '0.04',\n",
       "  'I9': '0.362',\n",
       "  'clicked': '0'}]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import csv\n",
    "import numpy as np\n",
    "\n",
    "with open('./data/train_eli5.csv', 'rt') as f:\n",
    "    data = list(csv.DictReader(f))\n",
    "data[:1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "899991 items total, 25.5% true\n"
     ]
    }
   ],
   "source": [
    "_all_xs = [{k: v for k, v in row.items() if k != 'clicked'} for row in data]\n",
    "_all_ys = np.array([int(row['clicked']) for row in data])\n",
    "\n",
    "all_xs, all_ys = shuffle(_all_xs, _all_ys, random_state=0)\n",
    "train_xs, valid_xs, train_ys, valid_ys = train_test_split(\n",
    "    all_xs, all_ys, test_size=0.25, random_state=0)\n",
    "print('{} items total, {:.1%} true'.format(len(all_xs), np.mean(all_ys)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.776 ± 0.003\n"
     ]
    }
   ],
   "source": [
    "# from xgboost import XGBClassifier\n",
    "import warnings\n",
    "# xgboost <= 0.6a2 shows a warning when used with scikit-learn 0.18+\n",
    "warnings.filterwarnings('ignore', category=UserWarning)\n",
    "class CSCTransformer:\n",
    "    def transform(self, xs):\n",
    "        # work around https://github.com/dmlc/xgboost/issues/1238#issuecomment-243872543\n",
    "        return xs.tocsc()\n",
    "    def fit(self, *args):\n",
    "        return self\n",
    "\n",
    "clf =  lgb.LGBMClassifier()\n",
    "vec = DictVectorizer()\n",
    "pipeline = make_pipeline(vec, CSCTransformer(), clf)\n",
    "\n",
    "def evaluate(_clf):\n",
    "    scores = cross_val_score(_clf, all_xs, all_ys, scoring='accuracy', cv=10)\n",
    "    print('Accuracy: {:.3f} ± {:.3f}'.format(np.mean(scores), 2 * np.std(scores)))\n",
    "    _clf.fit(train_xs, train_ys)  # so that parts of the original pipeline are fitted\n",
    "\n",
    "evaluate(pipeline)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "booster = clf.booster_   #如果运行出错请使用这句clf.booster()\n",
    "original_feature_names = booster.feature_name\n",
    "booster.feature_names = vec.get_feature_names()\n",
    "# recover original feature names\n",
    "booster.feature_names = original_feature_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <style>\n",
       "    table.eli5-weights tr:hover {\n",
       "        filter: brightness(85%);\n",
       "    }\n",
       "</style>\n",
       "\n",
       "\n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "        <table class=\"eli5-weights eli5-feature-importances\" style=\"border-collapse: collapse; border: none; margin-top: 0em; table-layout: auto;\">\n",
       "    <thead>\n",
       "    <tr style=\"border: none;\">\n",
       "        <th style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">Weight</th>\n",
       "        <th style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">Feature</th>\n",
       "    </tr>\n",
       "    </thead>\n",
       "    <tbody>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 80.00%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.1726\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I1=0\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 90.56%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0591\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I11=0\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 90.62%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0585\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I11=0.1\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 91.81%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0482\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I13=0\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 93.72%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0330\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I6=0\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 94.58%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0268\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I7=0\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 95.00%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0238\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                C17=18436\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 95.07%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0234\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                C14=14963\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 96.19%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0162\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                C14=14960\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 96.19%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0162\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                C14=14966\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 96.56%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0139\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I4=0\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 96.57%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0139\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I11=0.2\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 96.64%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0135\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                C23=21553\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 96.67%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0133\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I6=0.002\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 96.80%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0126\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I8=0\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 97.10%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0109\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I6=0.004\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 97.36%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0096\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                I13=0.02\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 97.50%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0089\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                C7=3696\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 97.59%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0084\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                C14=14967\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "        <tr style=\"background-color: hsl(120, 100.00%, 97.60%); border: none;\">\n",
       "            <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "                0.0084\n",
       "                \n",
       "            </td>\n",
       "            <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "                C20=20104\n",
       "            </td>\n",
       "        </tr>\n",
       "    \n",
       "    \n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 97.60%); border: none;\">\n",
       "                <td colspan=\"2\" style=\"padding: 0 0.5em 0 0.5em; text-align: center; border: none; white-space: nowrap;\">\n",
       "                    <i>&hellip; 102771 more &hellip;</i>\n",
       "                </td>\n",
       "            </tr>\n",
       "        \n",
       "    \n",
       "    </tbody>\n",
       "</table>\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from eli5 import show_weights\n",
    "show_weights(clf, vec=vec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <style>\n",
       "    table.eli5-weights tr:hover {\n",
       "        filter: brightness(85%);\n",
       "    }\n",
       "</style>\n",
       "\n",
       "\n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "        \n",
       "\n",
       "    \n",
       "\n",
       "        \n",
       "            \n",
       "                \n",
       "                \n",
       "    \n",
       "        <p style=\"margin-bottom: 0.5em; margin-top: 0em\">\n",
       "            <b>\n",
       "    \n",
       "        y=0\n",
       "    \n",
       "</b>\n",
       "\n",
       "    \n",
       "    (probability <b>0.931</b>, score <b>-2.597</b>)\n",
       "\n",
       "top features\n",
       "        </p>\n",
       "    \n",
       "    <table class=\"eli5-weights\"\n",
       "           style=\"border-collapse: collapse; border: none; margin-top: 0em; table-layout: auto; margin-bottom: 2em;\">\n",
       "        <thead>\n",
       "        <tr style=\"border: none;\">\n",
       "            \n",
       "                <th style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\" title=\"Feature contribution already accounts for the feature value (for linear models, contribution = weight * feature value), and the sum of feature contributions is equal to the score or, for some classifiers, to the probability. Feature values are shown if &quot;show_feature_values&quot; is True.\">\n",
       "                    Contribution<sup>?</sup>\n",
       "                </th>\n",
       "            \n",
       "            <th style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">Feature</th>\n",
       "            \n",
       "                <th style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">Value</th>\n",
       "            \n",
       "        </tr>\n",
       "        </thead>\n",
       "        <tbody>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 80.00%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +1.253\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        &lt;BIAS&gt;\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 93.10%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.274\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I11=0\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 94.38%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.205\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C9=6966\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 94.65%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.190\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I1=0\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 95.69%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.140\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I7=0\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 95.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.128\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C17=18436\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 97.01%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.083\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C20=20104\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 97.61%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.060\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I13=0\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 97.69%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.057\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C23=21548\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 97.80%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.054\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C25=23114\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 97.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.047\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21564\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 98.40%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.034\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I3=0\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 98.47%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.032\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C22=21538\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 98.60%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.028\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C9=6967\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 98.77%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.023\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14963\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 98.79%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.023\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I3=0.01\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 98.79%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.023\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I8=0\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 98.79%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.023\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C23=21549\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 98.89%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.020\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14961\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 98.89%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.020\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I8=0.02\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.00%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.017\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=137\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.14%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.014\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C23=21550\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.16%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.014\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I3=0.02\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.33%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.010\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C6=3688\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.35%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.009\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I4=0.02\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.44%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.008\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I13=0.06\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.44%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.008\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I8=0.04\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.52%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.006\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I3=0.04\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.52%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.006\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I3=0.03\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.53%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.006\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C20=20105\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.57%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.005\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I13=0.04\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.57%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.005\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C6=3685\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.58%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.005\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=3696\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.60%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.005\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C17=18442\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.60%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.005\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C19=19605\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.62%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I10=0\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.63%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11635\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.64%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18447\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.64%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.01\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.64%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.016\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.65%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.008\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.65%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.002\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.66%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13025\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.66%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.014\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.66%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.012\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.67%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.004\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.68%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I5=0.000016\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.68%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.018\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.68%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.006\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.68%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C20=20103\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.69%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I5=0.000031\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.69%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I3=0.05\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.70%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21567\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.70%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14962\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.70%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C19=19607\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.72%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15158\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.72%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I3=0.06\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.73%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23157\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.73%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C19=19626\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.74%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=163\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.74%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14965\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.74%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14968\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.75%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0.002\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.76%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I8=0.08\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.76%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13023\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.77%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I8=0.06\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.77%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23156\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.78%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I8=0.1\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.78%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.022\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.79%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.02\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.79%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9399\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.80%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13456\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.82%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=257\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.82%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0.006\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.82%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=177\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.82%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0.01\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.83%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.026\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.84%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C25=23125\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.84%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0.004\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.84%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0.008\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.84%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.84%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I10=0.25\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.84%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15304\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.85%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7015\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.85%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.024\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.85%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.032\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.85%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=162\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.86%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9400\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.86%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14979\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.86%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9719\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.86%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18456\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.87%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.028\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.87%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.036\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.87%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=150\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.87%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15025\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.88%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18594\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.88%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13060\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.89%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11644\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.89%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0.014\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.89%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9412\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.89%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C19=19610\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7002\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I5=0.000063\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15113\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23745\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18467\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18536\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C25=23122\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9417\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9382\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11815\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.034\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9436\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=186\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.91%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.04\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.91%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.03\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.91%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23169\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.91%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9418\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9457\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15072\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21617\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9434\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=14988\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23205\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0.02\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=1976\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9386\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=6990\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13143\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13200\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15104\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11638\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9420\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21734\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=16098\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15139\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18460\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23184\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21598\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=19048\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21589\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9717\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7001\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21584\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I11=1.2\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C16=17056\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13072\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18558\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=1977\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=180\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13338\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23196\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13140\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=317\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7023\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0.012\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I9=0.018\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9540\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18725\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=4425\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21641\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21627\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21699\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C16=17002\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9864\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9573\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23274\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13061\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13020\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12236\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21565\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9493\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7104\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12076\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9685\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.042\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I11=1.3\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15029\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9519\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9515\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23266\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13397\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9524\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=2091\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21626\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11671\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23292\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0.044\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18545\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=19364\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7239\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15195\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11911\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12292\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13125\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=215\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11691\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C21=20183\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11741\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11692\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13100\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15028\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9472\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21732\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23312\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15396\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=2198\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I11=1\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9544\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11641\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15379\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11702\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7043\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12194\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9841\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11745\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I5=0.000125\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15203\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18631\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=6985\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=5513\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9559\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12874\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15250\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18653\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18708\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21893\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11713\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15410\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=1950\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9756\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21662\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23283\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C19=19614\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18852\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11818\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21796\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15081\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I11=1.6\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=2559\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=10281\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C25=23130\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11791\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C25=23132\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15518\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C19=19632\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18610\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23446\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=3131\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21897\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=2680\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=248\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9530\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=3990\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=22034\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18801\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7096\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23521\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C3=1144\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9624\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18739\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9702\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23254\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15270\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13305\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21784\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15439\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23238\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21964\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15352\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15421\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C16=17597\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15487\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18786\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=4186\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15367\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=2166\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7319\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=2175\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18836\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15718\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=16516\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9858\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=16160\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=10217\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23429\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21848\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15569\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11898\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=422\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13781\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=22062\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=10499\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15872\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C21=20517\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12324\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12982\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15668\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=2425\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23516\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23580\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12148\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12135\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=2511\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=14745\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12926\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C16=17799\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=8762\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12377\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C16=18054\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=22934\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(120, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        +0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=19500\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "        \n",
       "\n",
       "        \n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=10546\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23481\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=12006\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C16=17265\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=10037\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23379\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15482\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11873\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7241\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21741\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13460\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21725\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.99%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7164\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11782\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9608\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13298\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21672\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13250\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=3881\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13148\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21656\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I13=0.82\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C19=19635\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9468\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9492\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9463\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15498\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13577\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15511\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=3863\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=229\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18604\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21755\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11950\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C19=19676\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15070\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.98%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I13=0.62\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15562\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23240\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18881\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=213\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13053\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13048\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23177\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23261\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18517\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9388\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13062\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=205\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13231\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=10060\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9907\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=6979\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=4057\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9512\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18662\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18491\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=3716\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9693\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21649\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=7016\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=4106\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.97%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9538\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18496\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15053\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9549\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11670\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=1920\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15003\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=3724\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=6973\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23250\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15476\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.96%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=14993\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13159\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11716\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21616\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18668\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23204\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=173\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13129\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13007\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23158\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9363\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21671\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9377\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11781\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21579\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21583\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15078\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9476\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9429\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11715\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I12=0.1\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21585\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13011\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.94%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18528\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11646\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15019\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=149\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11662\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C15=15305\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C19=19617\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.93%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=165\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13029\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21618\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=1927\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9381\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.000\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=169\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.92%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18488\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.91%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23163\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.91%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9349\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.91%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C17=18444\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.91%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=211\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.91%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14978\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11647\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18532\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I2=0.303987\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I2=0.008306\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C26=23179\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21594\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14975\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.90%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C12=11669\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.89%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C18=18461\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.89%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C17=18443\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.89%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=3693\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.88%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C11=9374\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.88%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I2=0.302326\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.86%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=6976\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.86%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=1908\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.86%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I1=0.05\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.85%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13016\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.84%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=155\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.84%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.001\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=156\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.81%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C7=3781\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.81%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=6974\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.80%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C16=16945\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.80%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C23=21556\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.80%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I7=0.01\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.79%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I2=0.006645\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.78%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I10=0.125\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.78%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.002\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C13=13010\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.73%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I2=0.004983\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.70%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=168\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.68%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.003\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C4=1897\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.67%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=6972\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.66%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=139\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.62%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.004\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I11=0.2\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.59%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.005\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I2=0.003322\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.58%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.005\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14964\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.57%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.005\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21568\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.55%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.006\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C17=18438\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.54%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.006\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C2=141\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.51%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.006\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C24=21575\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.47%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.007\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C25=23113\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.45%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.007\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C17=18440\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.40%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.008\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14967\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.34%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.010\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I11=0.1\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 99.02%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.017\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14966\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 98.85%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.021\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C10=6970\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 98.74%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.024\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C6=3683\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 98.53%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.030\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C23=21553\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 97.95%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.048\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        C14=14960\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 97.76%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.055\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I13=0.02\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            1.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "            <tr style=\"background-color: hsl(0, 100.00%, 96.76%); border: none;\">\n",
       "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
       "        -0.093\n",
       "    </td>\n",
       "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
       "        I6=0\n",
       "    </td>\n",
       "    \n",
       "        <td style=\"padding: 0 0.5em 0 1em; text-align: right; border: none;\">\n",
       "            0.000\n",
       "        </td>\n",
       "    \n",
       "</tr>\n",
       "        \n",
       "\n",
       "        </tbody>\n",
       "    </table>\n",
       "\n",
       "            \n",
       "        \n",
       "\n",
       "        \n",
       "\n",
       "\n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "    \n",
       "\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from eli5 import show_prediction\n",
    "show_prediction(clf, valid_xs[1], vec=vec, show_feature_values=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 用LightGBM的输出生成FM数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据格式请参见[`libFM 1.4.2 manual`](http://www.libfm.org/libfm-1.42.manual.pdf)中的说明，截取文档中的格式说明如下：\n",
    "<img src=\"fm_format.png\"\\>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GBDT已经训练好了，我们需要GBDT输出的叶子节点作为输入数据X传给FM，一共30个叶子节点，那么输入给FM的数据格式就是X中不是0的数据的index:value。\n",
    "\n",
    "一段真实数据如下：0\t0:31\t1:61\t2:93\t3:108\t4:149\t5:182\t6:212\t7:242\t8:277\t9:310\t10:334\t11:365\t12:401\t13:434\t14:465\t15:491\t16:527\t17:552\t18:589\t19:619\t20:648\t21:678\t22:697\t23:744\t24:770\t25:806\t26:826\t27:862\t28:899\t29:928\t30:955\t31:988"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generat_lgb2fm_data(outdir, gbm, dump, tr_path, va_path, te_path, _sep = '\\t'):\n",
    "    with open(os.path.join(outdir, 'train_lgb2fm.txt'), 'w') as out_train:\n",
    "        with open(os.path.join(outdir, 'valid_lgb2fm.txt'), 'w') as out_valid:\n",
    "            with open(os.path.join(outdir, 'test_lgb2fm.txt'), 'w') as out_test:\n",
    "                df_train_ = pd.read_csv(tr_path, header=None, sep=_sep)\n",
    "                df_valid_ = pd.read_csv(va_path, header=None, sep=_sep)\n",
    "                df_test_= pd.read_csv(te_path, header=None, sep=_sep)\n",
    "\n",
    "                y_train_ = df_train_[0].values\n",
    "                y_valid_ = df_valid_[0].values                \n",
    "\n",
    "                X_train_ = df_train_.drop(0, axis=1).values\n",
    "                X_valid_ = df_valid_.drop(0, axis=1).values\n",
    "                X_test_= df_test_.values\n",
    "   \n",
    "                train_leaves= gbm.predict(X_train_, num_iteration=gbm.best_iteration, pred_leaf=True)\n",
    "                valid_leaves= gbm.predict(X_valid_, num_iteration=gbm.best_iteration, pred_leaf=True)\n",
    "                test_leaves= gbm.predict(X_test_, num_iteration=gbm.best_iteration, pred_leaf=True)\n",
    "\n",
    "                tree_info = dump['tree_info']\n",
    "                tree_counts = len(tree_info)\n",
    "                for i in range(tree_counts):\n",
    "                    train_leaves[:, i] = train_leaves[:, i] + tree_info[i]['num_leaves'] * i + 1\n",
    "                    valid_leaves[:, i] = valid_leaves[:, i] + tree_info[i]['num_leaves'] * i + 1\n",
    "                    test_leaves[:, i] = test_leaves[:, i] + tree_info[i]['num_leaves'] * i + 1\n",
    "#                     print(train_leaves[:, i])\n",
    "#                     print(tree_info[i]['num_leaves'])\n",
    "\n",
    "                for idx in range(len(y_train_)):            \n",
    "                    out_train.write((str(y_train_[idx]) + '\\t'))\n",
    "                    out_train.write('\\t'.join(\n",
    "                        ['{}:{}'.format(ii, val) for ii,val in enumerate(train_leaves[idx]) if float(val) != 0 ]) + '\\n')\n",
    "                    \n",
    "                for idx in range(len(y_valid_)):                   \n",
    "                    out_valid.write((str(y_valid_[idx]) + '\\t'))\n",
    "                    out_valid.write('\\t'.join(\n",
    "                        ['{}:{}'.format(ii, val) for ii,val in enumerate(valid_leaves[idx]) if float(val) != 0 ]) + '\\n')\n",
    "                    \n",
    "                for idx in range(len(X_test_)):                   \n",
    "                    out_test.write('\\t'.join(\n",
    "                        ['{}:{}'.format(ii, val) for ii,val in enumerate(test_leaves[idx]) if float(val) != 0 ]) + '\\n')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "generat_lgb2fm_data('./data', gbm, dump, './data/train_lgb.txt', './data/valid_lgb.txt', './data/test_lgb.txt', '\\t')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练FM\n",
    "为训练FM的数据已经准备好了，我们调用LibFM进行训练。\n",
    "\n",
    "迭代64次，使用sgd训练，学习率是0.00000001，训练好的模型保存为文件fm_model。\n",
    "\n",
    "训练输出的log，Train和Test的数值不是loss，是accuracy。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['----------------------------------------------------------------------------\\n',\n",
       " 'libFM\\n',\n",
       " '  Version: 1.4.4\\n',\n",
       " '  Author:  Steffen Rendle, srendle@libfm.org\\n',\n",
       " '  WWW:     http://www.libfm.org/\\n',\n",
       " 'This program comes with ABSOLUTELY NO WARRANTY; for details see license.txt.\\n',\n",
       " 'This is free software, and you are welcome to redistribute it under certain\\n',\n",
       " 'conditions; for details see license.txt.\\n',\n",
       " '----------------------------------------------------------------------------\\n',\n",
       " 'Loading train...\\t\\n',\n",
       " 'has x = 1\\n',\n",
       " 'has xt = 0\\n',\n",
       " 'num_rows=899991\\tnum_values=28799712\\tnum_features=32\\tmin_target=0\\tmax_target=1\\n',\n",
       " 'Loading test... \\t\\n',\n",
       " 'has x = 1\\n',\n",
       " 'has xt = 0\\n',\n",
       " 'num_rows=100009\\tnum_values=3200288\\tnum_features=32\\tmin_target=0\\tmax_target=1\\n',\n",
       " '#relations: 0\\n',\n",
       " 'Loading meta data...\\t\\n',\n",
       " 'learnrate=1e-08\\n',\n",
       " 'learnrates=1e-08,1e-08,1e-08\\n',\n",
       " '#iterations=64\\n',\n",
       " \"SGD: DON'T FORGET TO SHUFFLE THE ROWS IN TRAINING DATA TO GET THE BEST RESULTS.\\n\",\n",
       " '#Iter=  0\\tTrain=0.625438\\tTest=0.619484\\n',\n",
       " '#Iter=  1\\tTrain=0.636596\\tTest=0.632013\\n',\n",
       " '#Iter=  2\\tTrain=0.627663\\tTest=0.623114\\n',\n",
       " '#Iter=  3\\tTrain=0.609776\\tTest=0.606605\\n',\n",
       " '#Iter=  4\\tTrain=0.563581\\tTest=0.56092\\n',\n",
       " '#Iter=  5\\tTrain=0.497907\\tTest=0.495655\\n',\n",
       " '#Iter=  6\\tTrain=0.461677\\tTest=0.461408\\n',\n",
       " '#Iter=  7\\tTrain=0.453666\\tTest=0.452639\\n',\n",
       " '#Iter=  8\\tTrain=0.454026\\tTest=0.453419\\n',\n",
       " '#Iter=  9\\tTrain=0.456836\\tTest=0.455919\\n',\n",
       " '#Iter= 10\\tTrain=0.46032\\tTest=0.459339\\n',\n",
       " '#Iter= 11\\tTrain=0.466546\\tTest=0.465358\\n',\n",
       " '#Iter= 12\\tTrain=0.473565\\tTest=0.472317\\n',\n",
       " '#Iter= 13\\tTrain=0.481726\\tTest=0.480967\\n',\n",
       " '#Iter= 14\\tTrain=0.492357\\tTest=0.491216\\n',\n",
       " '#Iter= 15\\tTrain=0.504419\\tTest=0.502935\\n',\n",
       " '#Iter= 16\\tTrain=0.517793\\tTest=0.516214\\n',\n",
       " '#Iter= 17\\tTrain=0.533604\\tTest=0.532102\\n',\n",
       " '#Iter= 18\\tTrain=0.552926\\tTest=0.5515\\n',\n",
       " '#Iter= 19\\tTrain=0.575645\\tTest=0.573198\\n',\n",
       " '#Iter= 20\\tTrain=0.59418\\tTest=0.590887\\n',\n",
       " '#Iter= 21\\tTrain=0.610691\\tTest=0.607815\\n',\n",
       " '#Iter= 22\\tTrain=0.626138\\tTest=0.623384\\n',\n",
       " '#Iter= 23\\tTrain=0.640751\\tTest=0.637923\\n',\n",
       " '#Iter= 24\\tTrain=0.65393\\tTest=0.652141\\n',\n",
       " '#Iter= 25\\tTrain=0.666099\\tTest=0.6641\\n',\n",
       " '#Iter= 26\\tTrain=0.677933\\tTest=0.675419\\n',\n",
       " '#Iter= 27\\tTrain=0.689539\\tTest=0.687108\\n',\n",
       " '#Iter= 28\\tTrain=0.700177\\tTest=0.697397\\n',\n",
       " '#Iter= 29\\tTrain=0.709265\\tTest=0.706156\\n',\n",
       " '#Iter= 30\\tTrain=0.716553\\tTest=0.713266\\n',\n",
       " '#Iter= 31\\tTrain=0.723218\\tTest=0.719635\\n',\n",
       " '#Iter= 32\\tTrain=0.729163\\tTest=0.726065\\n',\n",
       " '#Iter= 33\\tTrain=0.734428\\tTest=0.731354\\n',\n",
       " '#Iter= 34\\tTrain=0.738863\\tTest=0.735844\\n',\n",
       " '#Iter= 35\\tTrain=0.74284\\tTest=0.740323\\n',\n",
       " '#Iter= 36\\tTrain=0.746316\\tTest=0.743793\\n',\n",
       " '#Iter= 37\\tTrain=0.749123\\tTest=0.746333\\n',\n",
       " '#Iter= 38\\tTrain=0.751573\\tTest=0.748493\\n',\n",
       " '#Iter= 39\\tTrain=0.753264\\tTest=0.750292\\n',\n",
       " '#Iter= 40\\tTrain=0.754803\\tTest=0.751642\\n',\n",
       " '#Iter= 41\\tTrain=0.756011\\tTest=0.753062\\n',\n",
       " '#Iter= 42\\tTrain=0.756902\\tTest=0.753892\\n',\n",
       " '#Iter= 43\\tTrain=0.757642\\tTest=0.754872\\n',\n",
       " '#Iter= 44\\tTrain=0.758293\\tTest=0.755372\\n',\n",
       " '#Iter= 45\\tTrain=0.758855\\tTest=0.755782\\n',\n",
       " '#Iter= 46\\tTrain=0.759293\\tTest=0.756322\\n',\n",
       " '#Iter= 47\\tTrain=0.759695\\tTest=0.756652\\n',\n",
       " '#Iter= 48\\tTrain=0.760084\\tTest=0.756982\\n',\n",
       " '#Iter= 49\\tTrain=0.760343\\tTest=0.757252\\n',\n",
       " '#Iter= 50\\tTrain=0.76055\\tTest=0.757332\\n',\n",
       " '#Iter= 51\\tTrain=0.760706\\tTest=0.757582\\n',\n",
       " '#Iter= 52\\tTrain=0.760944\\tTest=0.757842\\n',\n",
       " '#Iter= 53\\tTrain=0.761035\\tTest=0.757952\\n',\n",
       " '#Iter= 54\\tTrain=0.761173\\tTest=0.758152\\n',\n",
       " '#Iter= 55\\tTrain=0.761291\\tTest=0.758382\\n',\n",
       " '#Iter= 56\\tTrain=0.76142\\tTest=0.758412\\n',\n",
       " '#Iter= 57\\tTrain=0.761541\\tTest=0.758452\\n',\n",
       " '#Iter= 58\\tTrain=0.761677\\tTest=0.758572\\n',\n",
       " '#Iter= 59\\tTrain=0.76175\\tTest=0.758692\\n',\n",
       " '#Iter= 60\\tTrain=0.761829\\tTest=0.758822\\n',\n",
       " '#Iter= 61\\tTrain=0.761855\\tTest=0.758862\\n',\n",
       " '#Iter= 62\\tTrain=0.761918\\tTest=0.759002\\n',\n",
       " '#Iter= 63\\tTrain=0.761988\\tTest=0.758972\\n',\n",
       " 'Final\\tTrain=0.761988\\tTest=0.758972\\n',\n",
       " 'Writing FM model to fm_model\\n']"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cmd = './libfm/libfm/bin/libFM -task c -train ./data/train_lgb2fm.txt -test ./data/valid_lgb2fm.txt -dim ’1,1,8’ -iter 64 -method sgd -learn_rate 0.00000001 -regular ’0,0,0.01’ -init_stdev 0.1 -save_model fm_model'\n",
    "os.popen(cmd).readlines()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "FM模型训练好了，我们把训练、验证和测试数据输入给FM，得到FM层的输出，输出的文件名为*.fm.logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['----------------------------------------------------------------------------\\n',\n",
       " 'libFM\\n',\n",
       " '  Version: 1.4.4\\n',\n",
       " '  Author:  Steffen Rendle, srendle@libfm.org\\n',\n",
       " '  WWW:     http://www.libfm.org/\\n',\n",
       " 'This program comes with ABSOLUTELY NO WARRANTY; for details see license.txt.\\n',\n",
       " 'This is free software, and you are welcome to redistribute it under certain\\n',\n",
       " 'conditions; for details see license.txt.\\n',\n",
       " '----------------------------------------------------------------------------\\n',\n",
       " 'Loading train...\\t\\n',\n",
       " 'has x = 1\\n',\n",
       " 'has xt = 0\\n',\n",
       " 'num_rows=899991\\tnum_values=28799712\\tnum_features=32\\tmin_target=0\\tmax_target=1\\n',\n",
       " 'Loading test... \\t\\n',\n",
       " 'has x = 1\\n',\n",
       " 'has xt = 0\\n',\n",
       " 'num_rows=100009\\tnum_values=3200288\\tnum_features=32\\tmin_target=0\\tmax_target=1\\n',\n",
       " '#relations: 0\\n',\n",
       " 'Loading meta data...\\t\\n',\n",
       " 'Reading FM model... \\t\\n',\n",
       " 'Final\\tTrain=0.761987\\tTest=0.758982\\n']"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cmd = './libfm/libfm/bin/libFM -task c -train ./data/train_lgb2fm.txt -test ./data/valid_lgb2fm.txt -dim ’1,1,8’ -iter 32 -method sgd -learn_rate 0.00000001 -regular ’0,0,0.01’ -init_stdev 0.1 -load_model fm_model -train_off true -prefix tr'\n",
    "os.popen(cmd).readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['----------------------------------------------------------------------------\\n',\n",
       " 'libFM\\n',\n",
       " '  Version: 1.4.4\\n',\n",
       " '  Author:  Steffen Rendle, srendle@libfm.org\\n',\n",
       " '  WWW:     http://www.libfm.org/\\n',\n",
       " 'This program comes with ABSOLUTELY NO WARRANTY; for details see license.txt.\\n',\n",
       " 'This is free software, and you are welcome to redistribute it under certain\\n',\n",
       " 'conditions; for details see license.txt.\\n',\n",
       " '----------------------------------------------------------------------------\\n',\n",
       " 'Loading train...\\t\\n',\n",
       " 'has x = 1\\n',\n",
       " 'has xt = 0\\n',\n",
       " 'num_rows=100009\\tnum_values=3200288\\tnum_features=32\\tmin_target=0\\tmax_target=1\\n',\n",
       " 'Loading test... \\t\\n',\n",
       " 'has x = 1\\n',\n",
       " 'has xt = 0\\n',\n",
       " 'num_rows=100009\\tnum_values=3200288\\tnum_features=32\\tmin_target=0\\tmax_target=1\\n',\n",
       " '#relations: 0\\n',\n",
       " 'Loading meta data...\\t\\n',\n",
       " 'Reading FM model... \\t\\n',\n",
       " 'Final\\tTrain=0.758982\\tTest=0.758982\\n']"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cmd = './libfm/libfm/bin/libFM -task c -train ./data/valid_lgb2fm.txt -test ./data/valid_lgb2fm.txt -dim ’1,1,8’ -iter 32 -method sgd -learn_rate 0.00000001 -regular ’0,0,0.01’ -init_stdev 0.1 -load_model fm_model -train_off true -prefix va'\n",
    "os.popen(cmd).readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['----------------------------------------------------------------------------\\n',\n",
       " 'libFM\\n',\n",
       " '  Version: 1.4.4\\n',\n",
       " '  Author:  Steffen Rendle, srendle@libfm.org\\n',\n",
       " '  WWW:     http://www.libfm.org/\\n',\n",
       " 'This program comes with ABSOLUTELY NO WARRANTY; for details see license.txt.\\n',\n",
       " 'This is free software, and you are welcome to redistribute it under certain\\n',\n",
       " 'conditions; for details see license.txt.\\n',\n",
       " '----------------------------------------------------------------------------\\n',\n",
       " 'Loading train...\\t\\n',\n",
       " 'has x = 1\\n',\n",
       " 'has xt = 0\\n',\n",
       " 'num_rows=1000000\\tnum_values=32000000\\tnum_features=32\\tmin_target=0\\tmax_target=1\\n',\n",
       " 'Loading test... \\t\\n',\n",
       " 'has x = 1\\n',\n",
       " 'has xt = 0\\n',\n",
       " 'num_rows=100009\\tnum_values=3200288\\tnum_features=32\\tmin_target=0\\tmax_target=1\\n',\n",
       " '#relations: 0\\n',\n",
       " 'Loading meta data...\\t\\n',\n",
       " 'Reading FM model... \\t\\n',\n",
       " 'Final\\tTest=0.758982\\n']"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cmd = './libfm/libfm/bin/libFM -task c -train ./data/test_lgb2fm.txt -test ./data/valid_lgb2fm.txt -dim ’1,1,8’ -iter 32 -method sgd -learn_rate 0.00000001 -regular ’0,0,0.01’ -init_stdev 0.1 -load_model fm_model -train_off true -prefix te -test2predict true'\n",
    "os.popen(cmd).readlines()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 开始构建模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "embed_dim = 32\n",
    "sparse_max = 30000 # sparse_feature_dim = 117568\n",
    "sparse_dim = 26\n",
    "dense_dim = 13\n",
    "out_dim = 400"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 定义输入占位符"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "def get_inputs():\n",
    "    dense_input = tf.placeholder(tf.float32, [None, dense_dim], name=\"dense_input\")\n",
    "    sparse_input = tf.placeholder(tf.int32, [None, sparse_dim], name=\"sparse_input\")\n",
    "    FFM_input = tf.placeholder(tf.float32, [None, 1], name=\"FFM_input\")\n",
    "    FM_input = tf.placeholder(tf.float32, [None, 1], name=\"FM_input\")\n",
    "    \n",
    "    targets = tf.placeholder(tf.float32, [None, 1], name=\"targets\")\n",
    "    LearningRate = tf.placeholder(tf.float32, name = \"LearningRate\")\n",
    "    return dense_input, sparse_input, FFM_input, FM_input, targets, LearningRate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 输入类别特征，从嵌入层获得嵌入向量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_sparse_embedding(sparse_input):\n",
    "    with tf.name_scope(\"sparse_embedding\"):\n",
    "        sparse_embed_matrix = tf.Variable(tf.random_uniform([sparse_max, embed_dim], -1, 1), name = \"sparse_embed_matrix\")\n",
    "        sparse_embed_layer = tf.nn.embedding_lookup(sparse_embed_matrix, sparse_input, name = \"sparse_embed_layer\")\n",
    "        sparse_embed_layer = tf.reshape(sparse_embed_layer, [-1, sparse_dim * embed_dim])\n",
    "    return sparse_embed_layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 输入数值特征，和嵌入向量链接在一起经过三层全连接层"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_dnn_layer(dense_input, sparse_embed_layer):\n",
    "    with tf.name_scope(\"dnn_layer\"):\n",
    "        input_combine_layer = tf.concat([dense_input, sparse_embed_layer], 1)  #(?, 845 = 832 + 13)\n",
    "        fc1_layer = tf.layers.dense(input_combine_layer, out_dim, name = \"fc1_layer\", activation=tf.nn.relu)\n",
    "        fc2_layer = tf.layers.dense(fc1_layer, out_dim, name = \"fc2_layer\", activation=tf.nn.relu)\n",
    "        fc3_layer = tf.layers.dense(fc2_layer, out_dim, name = \"fc3_layer\", activation=tf.nn.relu)\n",
    "    return fc3_layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 构建计算图\n",
    "如前所述，将FFM和FM层的输出经过全连接层，再和数值特征、嵌入向量的三层全连接层的输出连接在一起，做Logistic回归。\n",
    "\n",
    "采用LogLoss损失，FtrlOptimizer优化损失。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf.reset_default_graph()\n",
    "train_graph = tf.Graph()\n",
    "with train_graph.as_default():\n",
    "    dense_input, sparse_input, FFM_input, FM_input, targets, lr = get_inputs()\n",
    "    sparse_embed_layer = get_sparse_embedding(sparse_input)\n",
    "    fc3_layer = get_dnn_layer(dense_input, sparse_embed_layer)\n",
    "\n",
    "    ffm_fc_layer = tf.layers.dense(FFM_input, 1, name = \"ffm_fc_layer\")\n",
    "    fm_fc_layer = tf.layers.dense(FM_input, 1, name = \"fm_fc_layer\")\n",
    "    feature_combine_layer = tf.concat([ffm_fc_layer, fm_fc_layer, fc3_layer], 1)  #(?, 402)\n",
    "\n",
    "    with tf.name_scope(\"inference\"):\n",
    "        logits = tf.layers.dense(feature_combine_layer, 1, name = \"logits_layer\")\n",
    "        pred = tf.nn.sigmoid(logits, name = \"prediction\")\n",
    "    \n",
    "    with tf.name_scope(\"loss\"):\n",
    "        # LogLoss损失，Logistic回归到点击率\n",
    "#         cost = tf.losses.sigmoid_cross_entropy(targets, logits )\n",
    "        sigmoid_cost = tf.nn.sigmoid_cross_entropy_with_logits(labels=targets, logits=logits, name = \"sigmoid_cost\")\n",
    "        logloss_cost = tf.losses.log_loss(labels=targets, predictions=pred)\n",
    "        cost = logloss_cost # + sigmoid_cost\n",
    "        loss = tf.reduce_mean(cost)\n",
    "    # 优化损失 \n",
    "#     train_op = tf.train.AdamOptimizer(lr).minimize(loss)  #cost\n",
    "    global_step = tf.Variable(0, name=\"global_step\", trainable=False)\n",
    "    optimizer = tf.train.FtrlOptimizer(lr)  #tf.train.FtrlOptimizer(lr)  AdamOptimizer\n",
    "    gradients = optimizer.compute_gradients(loss)  #cost\n",
    "    train_op = optimizer.apply_gradients(gradients, global_step=global_step)\n",
    "    \n",
    "    # Accuracy\n",
    "    with tf.name_scope(\"score\"):\n",
    "        correct_prediction = tf.equal(tf.to_float(pred > 0.5), targets)\n",
    "        accuracy = tf.reduce_mean(tf.to_float(correct_prediction), name=\"accuracy\")\n",
    "        \n",
    "#     auc, uop = tf.contrib.metrics.streaming_auc(pred, targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 超参\n",
    "数据量太大，我们只跑一个epoch。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Number of Epochs\n",
    "num_epochs = 1\n",
    "# Batch Size\n",
    "batch_size = 32\n",
    "\n",
    "# Learning Rate\n",
    "learning_rate = 0.01\n",
    "# Show stats for every n number of batches\n",
    "show_every_n_batches = 25\n",
    "\n",
    "save_dir = './save'\n",
    "\n",
    "ffm_tr_out_path = './tr_ffm.out.logit'\n",
    "ffm_va_out_path = './va_ffm.out.logit'\n",
    "fm_tr_out_path = './tr.fm.logits'\n",
    "fm_va_out_path = './va.fm.logits'\n",
    "train_path = './data/train.txt'\n",
    "valid_path = './data/valid.txt'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_batches(Xs, ys, batch_size):\n",
    "    for start in range(0, len(Xs), batch_size):\n",
    "        end = min(start + batch_size, len(Xs))\n",
    "        yield Xs[start:end], ys[start:end]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 读取FFM的输出"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "ffm_train = pd.read_csv(ffm_tr_out_path, header=None)    \n",
    "ffm_train = ffm_train[0].values\n",
    "\n",
    "ffm_valid = pd.read_csv(ffm_va_out_path, header=None)    \n",
    "ffm_valid = ffm_valid[0].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 读取FM的输出"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "fm_train = pd.read_csv(fm_tr_out_path, header=None)    \n",
    "fm_train = fm_train[0].values\n",
    "\n",
    "fm_valid = pd.read_csv(fm_va_out_path, header=None)    \n",
    "fm_valid = fm_valid[0].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 读取数据集\n",
    "\n",
    "将DNN数据和FM、FFM的输出数据读取出来，并连接在一起"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = pd.read_csv(train_path, header=None)    \n",
    "train_data = train_data.values\n",
    "\n",
    "valid_data = pd.read_csv(valid_path, header=None)    \n",
    "valid_data = valid_data.values\n",
    "\n",
    "cc_train = np.concatenate((ffm_train.reshape(-1, 1), fm_train.reshape(-1, 1), train_data), 1)\n",
    "cc_valid = np.concatenate((ffm_valid.reshape(-1, 1), fm_valid.reshape(-1, 1), valid_data), 1)\n",
    "\n",
    "np.random.shuffle(cc_train)\n",
    "np.random.shuffle(cc_valid)\n",
    "\n",
    "train_y = cc_train[:,-1]\n",
    "test_y = cc_valid[:,-1]\n",
    "\n",
    "train_X = cc_train[:,0:-1]\n",
    "test_X = cc_valid[:,0:-1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "来看训练数据和验证数据的平均点击率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.25485810413659693"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(train_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.25576698097171252"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(test_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda/envs/tensorflow1.0/lib/python3.5/site-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n",
      "/Applications/anaconda/envs/tensorflow1.0/lib/python3.5/site-packages/sklearn/learning_curve.py:22: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the functions are moved. This module will be removed in 0.20\n",
      "  DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split\n",
    "import time\n",
    "import datetime\n",
    "from sklearn.metrics import log_loss\n",
    "from sklearn.learning_curve import learning_curve\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def train_model(num_epochs):\n",
    "    losses = {'train':[], 'test':[]}\n",
    "    acc_lst = {'train':[], 'test':[]}\n",
    "    pred_lst = []\n",
    "\n",
    "    with tf.Session(graph=train_graph) as sess:\n",
    "        \n",
    "        \n",
    "        # Keep track of gradient values and sparsity\n",
    "        grad_summaries = []\n",
    "        for g, v in gradients:\n",
    "            if g is not None:\n",
    "                grad_hist_summary = tf.summary.histogram(\"{}/grad/hist\".format(v.name.replace(':', '_')), g)\n",
    "                sparsity_summary = tf.summary.scalar(\"{}/grad/sparsity\".format(v.name.replace(':', '_')), tf.nn.zero_fraction(g))\n",
    "                grad_summaries.append(grad_hist_summary)\n",
    "                grad_summaries.append(sparsity_summary)\n",
    "        grad_summaries_merged = tf.summary.merge(grad_summaries)\n",
    "            \n",
    "        # Output directory for models and summaries\n",
    "        timestamp = str(int(time.time()))\n",
    "        out_dir = os.path.abspath(os.path.join(os.path.curdir, \"runs\", timestamp))\n",
    "        print(\"Writing to {}\\n\".format(out_dir))\n",
    "         \n",
    "        # Summaries for loss and accuracy\n",
    "        loss_summary = tf.summary.scalar(\"loss\", loss)\n",
    "#         acc_summary = tf.scalar_summary(\"accuracy\", accuracy)\n",
    "         \n",
    "        # Train Summaries\n",
    "        train_summary_op = tf.summary.merge([loss_summary, grad_summaries_merged])\n",
    "        train_summary_dir = os.path.join(out_dir, \"summaries\", \"train\")\n",
    "        train_summary_writer = tf.summary.FileWriter(train_summary_dir, sess.graph)\n",
    "    \n",
    "        # Inference summaries\n",
    "        inference_summary_op = tf.summary.merge([loss_summary])\n",
    "        inference_summary_dir = os.path.join(out_dir, \"summaries\", \"inference\")\n",
    "        inference_summary_writer = tf.summary.FileWriter(inference_summary_dir, sess.graph)\n",
    "    \n",
    "        sess.run(tf.global_variables_initializer())\n",
    "        sess.run(tf.local_variables_initializer())\n",
    "        saver = tf.train.Saver()\n",
    "        for epoch_i in range(num_epochs):\n",
    "            \n",
    "            #将数据集分成训练集和测试集\n",
    "            train_batches = get_batches(train_X, train_y, batch_size)\n",
    "            test_batches = get_batches(test_X, test_y, batch_size)\n",
    "        \n",
    "            #训练的迭代，保存训练损失\n",
    "            for batch_i in range(len(train_X) // batch_size):\n",
    "                x, y = next(train_batches)\n",
    "    \n",
    "                feed = {\n",
    "                    dense_input: x.take([2,3,4,5,6,7,8,9,10,11,12,13,14],1),\n",
    "                    sparse_input: x.take([15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],1),\n",
    "                    FFM_input: np.reshape(x.take(0,1), [batch_size, 1]),\n",
    "                    FM_input: np.reshape(x.take(1,1), [batch_size, 1]),\n",
    "                    targets: np.reshape(y, [batch_size, 1]),\n",
    "                    lr: learning_rate}\n",
    "    #             _ = sess.run([train_op], feed)  #cost\n",
    "                step, train_loss, summaries, _, prediction, acc = sess.run(\n",
    "                    [global_step, loss, train_summary_op, train_op, pred, accuracy], feed)  #cost\n",
    "                \n",
    "                prediction = prediction.reshape(y.shape)\n",
    "                losses['train'].append(train_loss)\n",
    "\n",
    "                acc_lst['train'].append(acc)\n",
    "                train_summary_writer.add_summary(summaries, step)  #\n",
    "\n",
    "                if(np.mean(y) != 0):\n",
    "                    auc = metrics.roc_auc_score(y, prediction)\n",
    "                else:\n",
    "                    auc = -1\n",
    "                    \n",
    "                # Show every <show_every_n_batches> batches\n",
    "                if (epoch_i * (len(train_X) // batch_size) + batch_i) % show_every_n_batches == 0:\n",
    "                    time_str = datetime.datetime.now().isoformat()\n",
    "                    print('{}: Epoch {:>3} Batch {:>4}/{}   train_loss = {:.3f}  accuracy = {}  auc = {}'.format(\n",
    "                        time_str,\n",
    "                        epoch_i,\n",
    "                        batch_i,\n",
    "                        (len(train_X) // batch_size),\n",
    "                        train_loss,\n",
    "                        acc,\n",
    "                        auc))\n",
    "#                     print(metrics.classification_report(y, np.float32(prediction > 0.5)))\n",
    "                    \n",
    "            #使用测试数据的迭代\n",
    "            for batch_i  in range(len(test_X) // batch_size):\n",
    "                x, y = next(test_batches)\n",
    "                \n",
    "                feed = {\n",
    "                    dense_input: x.take([2,3,4,5,6,7,8,9,10,11,12,13,14],1),\n",
    "                    sparse_input: x.take([15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],1),\n",
    "                    FFM_input: np.reshape(x.take(0,1), [batch_size, 1]),\n",
    "                    FM_input: np.reshape(x.take(1,1), [batch_size, 1]),\n",
    "                    targets: np.reshape(y, [batch_size, 1]),\n",
    "                    lr: learning_rate}\n",
    "                # Get Prediction\n",
    "                step, test_loss, summaries, prediction, acc = sess.run(\n",
    "                    [global_step, loss, inference_summary_op, pred, accuracy], feed)  #cost\n",
    "    \n",
    "                #保存测试损失和准确率\n",
    "                prediction = prediction.reshape(y.shape)\n",
    "                losses['test'].append(test_loss)\n",
    "\n",
    "                acc_lst['test'].append(acc)\n",
    "                inference_summary_writer.add_summary(summaries, step)  #\n",
    "                pred_lst.append(prediction)\n",
    "\n",
    "                if(np.mean(y) != 0):\n",
    "                    auc = metrics.roc_auc_score(y, prediction)\n",
    "                else:\n",
    "                    auc = -1\n",
    "\n",
    "                time_str = datetime.datetime.now().isoformat()\n",
    "                if (epoch_i * (len(test_X) // batch_size) + batch_i) % show_every_n_batches == 0:\n",
    "                    print('{}: Epoch {:>3} Batch {:>4}/{}   test_loss = {:.3f}  accuracy = {}  auc = {}'.format(\n",
    "                        time_str,\n",
    "                        epoch_i,\n",
    "                        batch_i,\n",
    "                        (len(test_X) // batch_size),\n",
    "                        test_loss,\n",
    "                        acc,\n",
    "                        auc))\n",
    "                    print(metrics.classification_report(y, np.float32(prediction > 0.5)))\n",
    "\n",
    "        # Save Model\n",
    "        saver.save(sess, save_dir)  #, global_step=epoch_i\n",
    "        print('Model Trained and Saved')\n",
    "        save_params((losses, acc_lst, pred_lst, save_dir))\n",
    "    return losses, acc_lst, pred_lst, save_dir\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing to /Users/chengstone/Downloads/cn-deep-learning-master/ctr/runs/1514636600\n",
      "\n",
      "2017-12-30T20:23:26.418668: Epoch   0 Batch    0/28124   train_loss = 0.693  accuracy = 0.5625  auc = 0.6145833333333334\n",
      "2017-12-30T20:23:27.780819: Epoch   0 Batch   25/28124   train_loss = 0.679  accuracy = 0.6875  auc = 0.9181818181818181\n",
      "2017-12-30T20:23:29.245611: Epoch   0 Batch   50/28124   train_loss = 0.676  accuracy = 0.65625  auc = 0.8051948051948052\n",
      "2017-12-30T20:23:30.612865: Epoch   0 Batch   75/28124   train_loss = 0.657  accuracy = 0.75  auc = 0.7135416666666667\n",
      "2017-12-30T20:23:31.944737: Epoch   0 Batch  100/28124   train_loss = 0.658  accuracy = 0.6875  auc = 0.8999999999999999\n",
      "2017-12-30T20:23:33.296840: Epoch   0 Batch  125/28124   train_loss = 0.622  accuracy = 0.84375  auc = 0.725925925925926\n",
      "2017-12-30T20:23:34.583400: Epoch   0 Batch  150/28124   train_loss = 0.609  accuracy = 0.84375  auc = 0.7999999999999999\n",
      "2017-12-30T20:23:35.881118: Epoch   0 Batch  175/28124   train_loss = 0.624  accuracy = 0.75  auc = 0.8385416666666666\n",
      "2017-12-30T20:23:37.228127: Epoch   0 Batch  200/28124   train_loss = 0.644  accuracy = 0.65625  auc = 0.7835497835497837\n",
      "2017-12-30T20:23:38.497814: Epoch   0 Batch  225/28124   train_loss = 0.616  accuracy = 0.71875  auc = 0.826086956521739\n",
      "2017-12-30T20:23:39.853496: Epoch   0 Batch  250/28124   train_loss = 0.563  accuracy = 0.84375  auc = 0.7851851851851852\n",
      "2017-12-30T20:23:41.183640: Epoch   0 Batch  275/28124   train_loss = 0.588  accuracy = 0.78125  auc = 0.6514285714285714\n",
      "2017-12-30T20:23:42.474078: Epoch   0 Batch  300/28124   train_loss = 0.621  accuracy = 0.65625  auc = 0.8268398268398269\n",
      "2017-12-30T20:23:43.766114: Epoch   0 Batch  325/28124   train_loss = 0.547  accuracy = 0.78125  auc = 0.9485714285714285\n",
      "2017-12-30T20:23:45.076048: Epoch   0 Batch  350/28124   train_loss = 0.655  accuracy = 0.5625  auc = 0.8571428571428572\n",
      "2017-12-30T20:23:46.345402: Epoch   0 Batch  375/28124   train_loss = 0.601  accuracy = 0.65625  auc = 0.8528138528138529\n",
      "2017-12-30T20:23:47.611929: Epoch   0 Batch  400/28124   train_loss = 0.524  accuracy = 0.8125  auc = 0.7884615384615385\n",
      "2017-12-30T20:23:48.924937: Epoch   0 Batch  425/28124   train_loss = 0.619  accuracy = 0.65625  auc = 0.6883116883116883\n",
      "2017-12-30T20:23:50.248906: Epoch   0 Batch  450/28124   train_loss = 0.537  accuracy = 0.75  auc = 0.84375\n",
      "2017-12-30T20:23:51.546762: Epoch   0 Batch  475/28124   train_loss = 0.570  accuracy = 0.6875  auc = 0.8727272727272728\n",
      "2017-12-30T20:23:52.829903: Epoch   0 Batch  500/28124   train_loss = 0.603  accuracy = 0.625  auc = 0.8458333333333333\n",
      "2017-12-30T20:23:54.151606: Epoch   0 Batch  525/28124   train_loss = 0.574  accuracy = 0.65625  auc = 0.8484848484848485\n",
      "2017-12-30T20:23:55.428096: Epoch   0 Batch  550/28124   train_loss = 0.671  accuracy = 0.59375  auc = 0.6437246963562753\n",
      "2017-12-30T20:23:56.714367: Epoch   0 Batch  575/28124   train_loss = 0.499  accuracy = 0.75  auc = 0.8958333333333333\n",
      "2017-12-30T20:23:58.017495: Epoch   0 Batch  600/28124   train_loss = 0.573  accuracy = 0.6875  auc = 0.7772727272727272\n",
      "2017-12-30T20:23:59.317653: Epoch   0 Batch  625/28124   train_loss = 0.419  accuracy = 0.875  auc = 0.6607142857142857\n",
      "2017-12-30T20:24:00.591958: Epoch   0 Batch  650/28124   train_loss = 0.562  accuracy = 0.6875  auc = 0.7909090909090909\n",
      "2017-12-30T20:24:01.868398: Epoch   0 Batch  675/28124   train_loss = 0.476  accuracy = 0.78125  auc = 0.8342857142857143\n",
      "2017-12-30T20:24:03.193729: Epoch   0 Batch  700/28124   train_loss = 0.484  accuracy = 0.8125  auc = 0.6987179487179487\n",
      "2017-12-30T20:24:04.430705: Epoch   0 Batch  725/28124   train_loss = 0.571  accuracy = 0.65625  auc = 0.8831168831168831\n",
      "2017-12-30T20:24:05.729278: Epoch   0 Batch  750/28124   train_loss = 0.427  accuracy = 0.84375  auc = 0.8222222222222223\n",
      "2017-12-30T20:24:07.067139: Epoch   0 Batch  775/28124   train_loss = 0.536  accuracy = 0.71875  auc = 0.7584541062801933\n",
      "2017-12-30T20:24:08.347595: Epoch   0 Batch  800/28124   train_loss = 0.593  accuracy = 0.6875  auc = 0.6727272727272727\n",
      "2017-12-30T20:24:09.639031: Epoch   0 Batch  825/28124   train_loss = 0.529  accuracy = 0.6875  auc = 0.8954545454545455\n",
      "2017-12-30T20:24:10.926139: Epoch   0 Batch  850/28124   train_loss = 0.546  accuracy = 0.6875  auc = 0.8999999999999999\n",
      "2017-12-30T20:24:12.278799: Epoch   0 Batch  875/28124   train_loss = 0.529  accuracy = 0.71875  auc = 0.7971014492753623\n",
      "2017-12-30T20:24:13.585784: Epoch   0 Batch  900/28124   train_loss = 0.501  accuracy = 0.75  auc = 0.828125\n",
      "2017-12-30T20:24:14.871460: Epoch   0 Batch  925/28124   train_loss = 0.675  accuracy = 0.59375  auc = 0.7698412698412698\n",
      "2017-12-30T20:24:16.204009: Epoch   0 Batch  950/28124   train_loss = 0.616  accuracy = 0.625  auc = 0.8208333333333333\n",
      "2017-12-30T20:24:17.503206: Epoch   0 Batch  975/28124   train_loss = 0.533  accuracy = 0.71875  auc = 0.782608695652174\n",
      "2017-12-30T20:24:18.786882: Epoch   0 Batch 1000/28124   train_loss = 0.601  accuracy = 0.65625  auc = 0.683982683982684\n",
      "2017-12-30T20:24:20.096825: Epoch   0 Batch 1025/28124   train_loss = 0.416  accuracy = 0.8125  auc = 0.9038461538461539\n",
      "2017-12-30T20:24:21.369606: Epoch   0 Batch 1050/28124   train_loss = 0.493  accuracy = 0.78125  auc = 0.6742857142857143\n",
      "2017-12-30T20:24:22.622502: Epoch   0 Batch 1075/28124   train_loss = 0.499  accuracy = 0.71875  auc = 0.9545454545454545\n",
      "2017-12-30T20:24:23.912033: Epoch   0 Batch 1100/28124   train_loss = 0.466  accuracy = 0.8125  auc = 0.6602564102564102\n",
      "2017-12-30T20:24:25.235344: Epoch   0 Batch 1125/28124   train_loss = 0.645  accuracy = 0.65625  auc = 0.645021645021645\n",
      "2017-12-30T20:24:26.513592: Epoch   0 Batch 1150/28124   train_loss = 0.461  accuracy = 0.78125  auc = 0.7657142857142857\n",
      "2017-12-30T20:24:27.786269: Epoch   0 Batch 1175/28124   train_loss = 0.468  accuracy = 0.75  auc = 0.8541666666666666\n",
      "2017-12-30T20:24:29.070042: Epoch   0 Batch 1200/28124   train_loss = 0.471  accuracy = 0.75  auc = 0.8489583333333334\n",
      "2017-12-30T20:24:30.333829: Epoch   0 Batch 1225/28124   train_loss = 0.699  accuracy = 0.59375  auc = 0.7301587301587302\n",
      "2017-12-30T20:24:31.601000: Epoch   0 Batch 1250/28124   train_loss = 0.414  accuracy = 0.8125  auc = 0.8653846153846154\n",
      "2017-12-30T20:24:32.854690: Epoch   0 Batch 1275/28124   train_loss = 0.610  accuracy = 0.65625  auc = 0.8208333333333333\n",
      "2017-12-30T20:24:34.160319: Epoch   0 Batch 1300/28124   train_loss = 0.393  accuracy = 0.8125  auc = 0.8846153846153847\n",
      "2017-12-30T20:24:35.417689: Epoch   0 Batch 1325/28124   train_loss = 0.440  accuracy = 0.78125  auc = 0.8628571428571428\n",
      "2017-12-30T20:24:36.705283: Epoch   0 Batch 1350/28124   train_loss = 0.613  accuracy = 0.625  auc = 0.9087301587301587\n",
      "2017-12-30T20:24:37.985912: Epoch   0 Batch 1375/28124   train_loss = 0.538  accuracy = 0.75  auc = 0.6458333333333334\n",
      "2017-12-30T20:24:39.299893: Epoch   0 Batch 1400/28124   train_loss = 0.475  accuracy = 0.71875  auc = 0.9227053140096618\n",
      "2017-12-30T20:24:40.563473: Epoch   0 Batch 1425/28124   train_loss = 0.439  accuracy = 0.84375  auc = 0.7051282051282051\n",
      "2017-12-30T20:24:41.826123: Epoch   0 Batch 1450/28124   train_loss = 0.536  accuracy = 0.71875  auc = 0.7101449275362318\n",
      "2017-12-30T20:24:43.127040: Epoch   0 Batch 1475/28124   train_loss = 0.482  accuracy = 0.75  auc = 0.7916666666666666\n",
      "2017-12-30T20:24:44.400981: Epoch   0 Batch 1500/28124   train_loss = 0.419  accuracy = 0.75  auc = 0.9583333333333334\n",
      "2017-12-30T20:24:45.657939: Epoch   0 Batch 1525/28124   train_loss = 0.609  accuracy = 0.625  auc = 0.8380566801619433\n",
      "2017-12-30T20:24:46.922224: Epoch   0 Batch 1550/28124   train_loss = 0.473  accuracy = 0.78125  auc = 0.8385416666666667\n",
      "2017-12-30T20:24:48.232617: Epoch   0 Batch 1575/28124   train_loss = 0.486  accuracy = 0.78125  auc = 0.7708333333333334\n",
      "2017-12-30T20:24:49.495434: Epoch   0 Batch 1600/28124   train_loss = 0.424  accuracy = 0.78125  auc = 0.88\n",
      "2017-12-30T20:24:50.826979: Epoch   0 Batch 1625/28124   train_loss = 0.519  accuracy = 0.75  auc = 0.7760416666666666\n",
      "2017-12-30T20:24:52.163689: Epoch   0 Batch 1650/28124   train_loss = 0.435  accuracy = 0.78125  auc = 0.9468599033816425\n",
      "2017-12-30T20:24:53.493656: Epoch   0 Batch 1675/28124   train_loss = 0.676  accuracy = 0.53125  auc = 0.8352941176470587\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:24:54.748792: Epoch   0 Batch 1700/28124   train_loss = 0.443  accuracy = 0.8125  auc = 0.7628205128205128\n",
      "2017-12-30T20:24:56.069509: Epoch   0 Batch 1725/28124   train_loss = 0.489  accuracy = 0.78125  auc = 0.7485714285714284\n",
      "2017-12-30T20:24:57.348857: Epoch   0 Batch 1750/28124   train_loss = 0.472  accuracy = 0.75  auc = 0.8020833333333334\n",
      "2017-12-30T20:24:58.596477: Epoch   0 Batch 1775/28124   train_loss = 0.403  accuracy = 0.8125  auc = 0.8589743589743589\n",
      "2017-12-30T20:24:59.860246: Epoch   0 Batch 1800/28124   train_loss = 0.408  accuracy = 0.8125  auc = 0.8461538461538463\n",
      "2017-12-30T20:25:01.145966: Epoch   0 Batch 1825/28124   train_loss = 0.491  accuracy = 0.71875  auc = 0.7447916666666667\n",
      "2017-12-30T20:25:02.394145: Epoch   0 Batch 1850/28124   train_loss = 0.426  accuracy = 0.78125  auc = 0.9270833333333333\n",
      "2017-12-30T20:25:03.696468: Epoch   0 Batch 1875/28124   train_loss = 0.442  accuracy = 0.75  auc = 0.9227053140096618\n",
      "2017-12-30T20:25:04.966691: Epoch   0 Batch 1900/28124   train_loss = 0.423  accuracy = 0.78125  auc = 0.8205128205128205\n",
      "2017-12-30T20:25:06.266366: Epoch   0 Batch 1925/28124   train_loss = 0.349  accuracy = 0.875  auc = 0.7767857142857143\n",
      "2017-12-30T20:25:07.517438: Epoch   0 Batch 1950/28124   train_loss = 0.480  accuracy = 0.8125  auc = 0.596153846153846\n",
      "2017-12-30T20:25:08.782927: Epoch   0 Batch 1975/28124   train_loss = 0.516  accuracy = 0.6875  auc = 0.8454106280193237\n",
      "2017-12-30T20:25:10.085529: Epoch   0 Batch 2000/28124   train_loss = 0.547  accuracy = 0.6875  auc = 0.8311688311688312\n",
      "2017-12-30T20:25:11.361690: Epoch   0 Batch 2025/28124   train_loss = 0.469  accuracy = 0.71875  auc = 0.8333333333333334\n",
      "2017-12-30T20:25:12.637559: Epoch   0 Batch 2050/28124   train_loss = 0.502  accuracy = 0.75  auc = 0.71875\n",
      "2017-12-30T20:25:13.940050: Epoch   0 Batch 2075/28124   train_loss = 0.475  accuracy = 0.75  auc = 0.855072463768116\n",
      "2017-12-30T20:25:15.227596: Epoch   0 Batch 2100/28124   train_loss = 0.541  accuracy = 0.75  auc = 0.8181818181818182\n",
      "2017-12-30T20:25:16.477694: Epoch   0 Batch 2125/28124   train_loss = 0.441  accuracy = 0.84375  auc = 0.7542857142857143\n",
      "2017-12-30T20:25:17.760595: Epoch   0 Batch 2150/28124   train_loss = 0.522  accuracy = 0.71875  auc = 0.7342995169082126\n",
      "2017-12-30T20:25:19.092378: Epoch   0 Batch 2175/28124   train_loss = 0.638  accuracy = 0.65625  auc = 0.7416666666666667\n",
      "2017-12-30T20:25:20.398024: Epoch   0 Batch 2200/28124   train_loss = 0.651  accuracy = 0.625  auc = 0.7583333333333333\n",
      "2017-12-30T20:25:21.740033: Epoch   0 Batch 2225/28124   train_loss = 0.329  accuracy = 0.8125  auc = 0.9407407407407408\n",
      "2017-12-30T20:25:23.067194: Epoch   0 Batch 2250/28124   train_loss = 0.545  accuracy = 0.65625  auc = 0.8181818181818181\n",
      "2017-12-30T20:25:24.344378: Epoch   0 Batch 2275/28124   train_loss = 0.382  accuracy = 0.90625  auc = 0.7185185185185186\n",
      "2017-12-30T20:25:25.600453: Epoch   0 Batch 2300/28124   train_loss = 0.444  accuracy = 0.75  auc = 0.8645833333333333\n",
      "2017-12-30T20:25:26.892922: Epoch   0 Batch 2325/28124   train_loss = 0.393  accuracy = 0.8125  auc = 0.903846153846154\n",
      "2017-12-30T20:25:28.202919: Epoch   0 Batch 2350/28124   train_loss = 0.501  accuracy = 0.65625  auc = 0.8571428571428571\n",
      "2017-12-30T20:25:29.463728: Epoch   0 Batch 2375/28124   train_loss = 0.397  accuracy = 0.78125  auc = 0.8205128205128205\n",
      "2017-12-30T20:25:30.708938: Epoch   0 Batch 2400/28124   train_loss = 0.377  accuracy = 0.8125  auc = 0.9102564102564102\n",
      "2017-12-30T20:25:31.969698: Epoch   0 Batch 2425/28124   train_loss = 0.407  accuracy = 0.8125  auc = 0.8525641025641026\n",
      "2017-12-30T20:25:33.272712: Epoch   0 Batch 2450/28124   train_loss = 0.517  accuracy = 0.71875  auc = 0.7874396135265701\n",
      "2017-12-30T20:25:34.537293: Epoch   0 Batch 2475/28124   train_loss = 0.493  accuracy = 0.75  auc = 0.7552083333333333\n",
      "2017-12-30T20:25:35.811169: Epoch   0 Batch 2500/28124   train_loss = 0.490  accuracy = 0.8125  auc = 0.6571428571428571\n",
      "2017-12-30T20:25:37.149680: Epoch   0 Batch 2525/28124   train_loss = 0.367  accuracy = 0.90625  auc = 0.7321428571428571\n",
      "2017-12-30T20:25:38.414199: Epoch   0 Batch 2550/28124   train_loss = 0.470  accuracy = 0.78125  auc = 0.7428571428571429\n",
      "2017-12-30T20:25:39.676961: Epoch   0 Batch 2575/28124   train_loss = 0.405  accuracy = 0.8125  auc = 0.8645833333333335\n",
      "2017-12-30T20:25:40.955865: Epoch   0 Batch 2600/28124   train_loss = 0.285  accuracy = 0.90625  auc = 0.9464285714285715\n",
      "2017-12-30T20:25:42.267454: Epoch   0 Batch 2625/28124   train_loss = 0.542  accuracy = 0.6875  auc = 0.753623188405797\n",
      "2017-12-30T20:25:43.533608: Epoch   0 Batch 2650/28124   train_loss = 0.365  accuracy = 0.84375  auc = 0.837037037037037\n",
      "2017-12-30T20:25:44.809237: Epoch   0 Batch 2675/28124   train_loss = 0.564  accuracy = 0.75  auc = 0.6859903381642511\n",
      "2017-12-30T20:25:46.100191: Epoch   0 Batch 2700/28124   train_loss = 0.441  accuracy = 0.84375  auc = 0.6923076923076923\n",
      "2017-12-30T20:25:47.358555: Epoch   0 Batch 2725/28124   train_loss = 0.460  accuracy = 0.78125  auc = 0.7257142857142856\n",
      "2017-12-30T20:25:48.609638: Epoch   0 Batch 2750/28124   train_loss = 0.405  accuracy = 0.8125  auc = 0.9082125603864734\n",
      "2017-12-30T20:25:49.862272: Epoch   0 Batch 2775/28124   train_loss = 0.455  accuracy = 0.78125  auc = 0.7885714285714285\n",
      "2017-12-30T20:25:51.190436: Epoch   0 Batch 2800/28124   train_loss = 0.306  accuracy = 0.9375  auc = 0.9185185185185185\n",
      "2017-12-30T20:25:52.457828: Epoch   0 Batch 2825/28124   train_loss = 0.383  accuracy = 0.8125  auc = 0.9028571428571429\n",
      "2017-12-30T20:25:53.717076: Epoch   0 Batch 2850/28124   train_loss = 0.458  accuracy = 0.8125  auc = 0.8502415458937198\n",
      "2017-12-30T20:25:54.997384: Epoch   0 Batch 2875/28124   train_loss = 0.393  accuracy = 0.84375  auc = 0.7756410256410257\n",
      "2017-12-30T20:25:56.320141: Epoch   0 Batch 2900/28124   train_loss = 0.506  accuracy = 0.71875  auc = 0.8772727272727273\n",
      "2017-12-30T20:25:57.580079: Epoch   0 Batch 2925/28124   train_loss = 0.379  accuracy = 0.8125  auc = 0.8782051282051282\n",
      "2017-12-30T20:25:58.836761: Epoch   0 Batch 2950/28124   train_loss = 0.434  accuracy = 0.78125  auc = 0.8697916666666667\n",
      "2017-12-30T20:26:00.170908: Epoch   0 Batch 2975/28124   train_loss = 0.528  accuracy = 0.75  auc = 0.7135416666666667\n",
      "2017-12-30T20:26:01.434311: Epoch   0 Batch 3000/28124   train_loss = 0.317  accuracy = 0.9375  auc = 0.9423076923076923\n",
      "2017-12-30T20:26:02.697156: Epoch   0 Batch 3025/28124   train_loss = 0.517  accuracy = 0.6875  auc = 0.7584541062801933\n",
      "2017-12-30T20:26:03.953277: Epoch   0 Batch 3050/28124   train_loss = 0.390  accuracy = 0.84375  auc = 0.7410714285714285\n",
      "2017-12-30T20:26:05.232427: Epoch   0 Batch 3075/28124   train_loss = 0.434  accuracy = 0.8125  auc = 0.5625\n",
      "2017-12-30T20:26:06.506853: Epoch   0 Batch 3100/28124   train_loss = 0.471  accuracy = 0.75  auc = 0.8072916666666666\n",
      "2017-12-30T20:26:07.772820: Epoch   0 Batch 3125/28124   train_loss = 0.457  accuracy = 0.78125  auc = 0.6987179487179487\n",
      "2017-12-30T20:26:09.075781: Epoch   0 Batch 3150/28124   train_loss = 0.489  accuracy = 0.78125  auc = 0.8136363636363636\n",
      "2017-12-30T20:26:10.312912: Epoch   0 Batch 3175/28124   train_loss = 0.381  accuracy = 0.8125  auc = 0.8971428571428571\n",
      "2017-12-30T20:26:11.566720: Epoch   0 Batch 3200/28124   train_loss = 0.406  accuracy = 0.8125  auc = 0.8971428571428571\n",
      "2017-12-30T20:26:12.843366: Epoch   0 Batch 3225/28124   train_loss = 0.366  accuracy = 0.875  auc = 0.8717948717948718\n",
      "2017-12-30T20:26:14.145575: Epoch   0 Batch 3250/28124   train_loss = 0.386  accuracy = 0.8125  auc = 0.921875\n",
      "2017-12-30T20:26:15.406080: Epoch   0 Batch 3275/28124   train_loss = 0.427  accuracy = 0.84375  auc = 0.7942857142857143\n",
      "2017-12-30T20:26:16.680564: Epoch   0 Batch 3300/28124   train_loss = 0.390  accuracy = 0.8125  auc = 0.84\n",
      "2017-12-30T20:26:17.955998: Epoch   0 Batch 3325/28124   train_loss = 0.301  accuracy = 0.875  auc = 0.9555555555555556\n",
      "2017-12-30T20:26:19.241314: Epoch   0 Batch 3350/28124   train_loss = 0.543  accuracy = 0.6875  auc = 0.7545454545454545\n",
      "2017-12-30T20:26:20.485711: Epoch   0 Batch 3375/28124   train_loss = 0.276  accuracy = 0.9375  auc = 1.0\n",
      "2017-12-30T20:26:21.750428: Epoch   0 Batch 3400/28124   train_loss = 0.855  accuracy = 0.5625  auc = 0.54296875\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:26:23.005343: Epoch   0 Batch 3425/28124   train_loss = 0.433  accuracy = 0.78125  auc = 0.8385416666666666\n",
      "2017-12-30T20:26:24.351599: Epoch   0 Batch 3450/28124   train_loss = 0.450  accuracy = 0.8125  auc = 0.49107142857142855\n",
      "2017-12-30T20:26:25.606429: Epoch   0 Batch 3475/28124   train_loss = 0.498  accuracy = 0.75  auc = 0.7681159420289855\n",
      "2017-12-30T20:26:26.892939: Epoch   0 Batch 3500/28124   train_loss = 0.529  accuracy = 0.71875  auc = 0.7954545454545454\n",
      "2017-12-30T20:26:28.209326: Epoch   0 Batch 3525/28124   train_loss = 0.488  accuracy = 0.71875  auc = 0.8164251207729469\n",
      "2017-12-30T20:26:29.462630: Epoch   0 Batch 3550/28124   train_loss = 0.637  accuracy = 0.625  auc = 0.7041666666666666\n",
      "2017-12-30T20:26:30.740970: Epoch   0 Batch 3575/28124   train_loss = 0.380  accuracy = 0.78125  auc = 0.858974358974359\n",
      "2017-12-30T20:26:32.027468: Epoch   0 Batch 3600/28124   train_loss = 0.371  accuracy = 0.8125  auc = 0.8592592592592593\n",
      "2017-12-30T20:26:33.314525: Epoch   0 Batch 3625/28124   train_loss = 0.391  accuracy = 0.84375  auc = 0.8457142857142856\n",
      "2017-12-30T20:26:34.573455: Epoch   0 Batch 3650/28124   train_loss = 0.522  accuracy = 0.71875  auc = 0.8115942028985507\n",
      "2017-12-30T20:26:35.842229: Epoch   0 Batch 3675/28124   train_loss = 0.614  accuracy = 0.65625  auc = 0.7142857142857143\n",
      "2017-12-30T20:26:37.176825: Epoch   0 Batch 3700/28124   train_loss = 0.400  accuracy = 0.78125  auc = 0.84\n",
      "2017-12-30T20:26:38.430951: Epoch   0 Batch 3725/28124   train_loss = 0.504  accuracy = 0.71875  auc = 0.8067632850241546\n",
      "2017-12-30T20:26:39.673777: Epoch   0 Batch 3750/28124   train_loss = 0.519  accuracy = 0.6875  auc = 0.9004329004329005\n",
      "2017-12-30T20:26:40.938586: Epoch   0 Batch 3775/28124   train_loss = 0.394  accuracy = 0.78125  auc = 0.9010416666666666\n",
      "2017-12-30T20:26:42.226670: Epoch   0 Batch 3800/28124   train_loss = 0.450  accuracy = 0.78125  auc = 0.8357487922705314\n",
      "2017-12-30T20:26:43.483238: Epoch   0 Batch 3825/28124   train_loss = 0.627  accuracy = 0.65625  auc = 0.7732793522267207\n",
      "2017-12-30T20:26:44.740672: Epoch   0 Batch 3850/28124   train_loss = 0.273  accuracy = 0.96875  auc = 0.9553571428571428\n",
      "2017-12-30T20:26:46.035437: Epoch   0 Batch 3875/28124   train_loss = 0.461  accuracy = 0.8125  auc = 0.7371428571428571\n",
      "2017-12-30T20:26:47.327444: Epoch   0 Batch 3900/28124   train_loss = 0.454  accuracy = 0.84375  auc = 0.4821428571428571\n",
      "2017-12-30T20:26:48.585794: Epoch   0 Batch 3925/28124   train_loss = 0.314  accuracy = 0.875  auc = 0.8839285714285714\n",
      "2017-12-30T20:26:49.839848: Epoch   0 Batch 3950/28124   train_loss = 0.441  accuracy = 0.78125  auc = 0.828125\n",
      "2017-12-30T20:26:51.200363: Epoch   0 Batch 3975/28124   train_loss = 0.483  accuracy = 0.84375  auc = 0.7874396135265701\n",
      "2017-12-30T20:26:52.474418: Epoch   0 Batch 4000/28124   train_loss = 0.692  accuracy = 0.65625  auc = 0.6904761904761905\n",
      "2017-12-30T20:26:53.799652: Epoch   0 Batch 4025/28124   train_loss = 0.291  accuracy = 0.96875  auc = 1.0\n",
      "2017-12-30T20:26:55.101407: Epoch   0 Batch 4050/28124   train_loss = 0.485  accuracy = 0.78125  auc = 0.6875\n",
      "2017-12-30T20:26:56.377339: Epoch   0 Batch 4075/28124   train_loss = 0.440  accuracy = 0.8125  auc = 0.8229166666666667\n",
      "2017-12-30T20:26:57.640029: Epoch   0 Batch 4100/28124   train_loss = 0.564  accuracy = 0.6875  auc = 0.8484848484848485\n",
      "2017-12-30T20:26:58.909106: Epoch   0 Batch 4125/28124   train_loss = 0.483  accuracy = 0.71875  auc = 0.8681818181818182\n",
      "2017-12-30T20:27:00.222011: Epoch   0 Batch 4150/28124   train_loss = 0.382  accuracy = 0.875  auc = 0.921875\n",
      "2017-12-30T20:27:01.450701: Epoch   0 Batch 4175/28124   train_loss = 0.427  accuracy = 0.8125  auc = 0.7657142857142858\n",
      "2017-12-30T20:27:02.704487: Epoch   0 Batch 4200/28124   train_loss = 0.518  accuracy = 0.75  auc = 0.7294685990338164\n",
      "2017-12-30T20:27:03.973866: Epoch   0 Batch 4225/28124   train_loss = 0.439  accuracy = 0.71875  auc = 0.932367149758454\n",
      "2017-12-30T20:27:05.249943: Epoch   0 Batch 4250/28124   train_loss = 0.488  accuracy = 0.75  auc = 0.8272727272727273\n",
      "2017-12-30T20:27:06.517118: Epoch   0 Batch 4275/28124   train_loss = 0.414  accuracy = 0.84375  auc = 0.7371794871794872\n",
      "2017-12-30T20:27:07.767827: Epoch   0 Batch 4300/28124   train_loss = 0.396  accuracy = 0.78125  auc = 0.8628571428571429\n",
      "2017-12-30T20:27:09.082634: Epoch   0 Batch 4325/28124   train_loss = 0.554  accuracy = 0.6875  auc = 0.8623481781376519\n",
      "2017-12-30T20:27:10.360225: Epoch   0 Batch 4350/28124   train_loss = 0.419  accuracy = 0.78125  auc = 0.890625\n",
      "2017-12-30T20:27:11.598832: Epoch   0 Batch 4375/28124   train_loss = 0.379  accuracy = 0.71875  auc = 0.9085714285714286\n",
      "2017-12-30T20:27:12.853327: Epoch   0 Batch 4400/28124   train_loss = 0.398  accuracy = 0.84375  auc = 0.8958333333333334\n",
      "2017-12-30T20:27:14.136706: Epoch   0 Batch 4425/28124   train_loss = 0.489  accuracy = 0.6875  auc = 0.8727272727272728\n",
      "2017-12-30T20:27:15.416802: Epoch   0 Batch 4450/28124   train_loss = 0.380  accuracy = 0.875  auc = 0.8\n",
      "2017-12-30T20:27:16.667015: Epoch   0 Batch 4475/28124   train_loss = 0.373  accuracy = 0.875  auc = 0.90625\n",
      "2017-12-30T20:27:17.968453: Epoch   0 Batch 4500/28124   train_loss = 0.417  accuracy = 0.78125  auc = 0.8840579710144928\n",
      "2017-12-30T20:27:19.272572: Epoch   0 Batch 4525/28124   train_loss = 0.548  accuracy = 0.65625  auc = 0.7727272727272727\n",
      "2017-12-30T20:27:20.529264: Epoch   0 Batch 4550/28124   train_loss = 0.455  accuracy = 0.90625  auc = 0.7142857142857143\n",
      "2017-12-30T20:27:21.795476: Epoch   0 Batch 4575/28124   train_loss = 0.676  accuracy = 0.6875  auc = 0.7041666666666666\n",
      "2017-12-30T20:27:23.119576: Epoch   0 Batch 4600/28124   train_loss = 0.488  accuracy = 0.75  auc = 0.7604166666666666\n",
      "2017-12-30T20:27:24.387382: Epoch   0 Batch 4625/28124   train_loss = 0.403  accuracy = 0.84375  auc = 0.7756410256410257\n",
      "2017-12-30T20:27:25.646423: Epoch   0 Batch 4650/28124   train_loss = 0.380  accuracy = 0.875  auc = 0.8514285714285714\n",
      "2017-12-30T20:27:26.945317: Epoch   0 Batch 4675/28124   train_loss = 0.538  accuracy = 0.78125  auc = 0.7439613526570048\n",
      "2017-12-30T20:27:28.239377: Epoch   0 Batch 4700/28124   train_loss = 0.416  accuracy = 0.84375  auc = 0.7942857142857143\n",
      "2017-12-30T20:27:29.492760: Epoch   0 Batch 4725/28124   train_loss = 0.403  accuracy = 0.84375  auc = 0.6888888888888889\n",
      "2017-12-30T20:27:30.770869: Epoch   0 Batch 4750/28124   train_loss = 0.475  accuracy = 0.75  auc = 0.875\n",
      "2017-12-30T20:27:32.061007: Epoch   0 Batch 4775/28124   train_loss = 0.451  accuracy = 0.78125  auc = 0.8309178743961353\n",
      "2017-12-30T20:27:33.326584: Epoch   0 Batch 4800/28124   train_loss = 0.454  accuracy = 0.84375  auc = 0.6685714285714285\n",
      "2017-12-30T20:27:34.577694: Epoch   0 Batch 4825/28124   train_loss = 0.562  accuracy = 0.71875  auc = 0.5542857142857143\n",
      "2017-12-30T20:27:35.850024: Epoch   0 Batch 4850/28124   train_loss = 0.392  accuracy = 0.84375  auc = 0.8697916666666667\n",
      "2017-12-30T20:27:37.178644: Epoch   0 Batch 4875/28124   train_loss = 0.549  accuracy = 0.71875  auc = 0.671875\n",
      "2017-12-30T20:27:38.430778: Epoch   0 Batch 4900/28124   train_loss = 0.501  accuracy = 0.75  auc = 0.8791666666666667\n",
      "2017-12-30T20:27:39.698728: Epoch   0 Batch 4925/28124   train_loss = 0.397  accuracy = 0.78125  auc = 0.9480519480519479\n",
      "2017-12-30T20:27:40.959328: Epoch   0 Batch 4950/28124   train_loss = 0.520  accuracy = 0.78125  auc = 0.7363636363636363\n",
      "2017-12-30T20:27:42.243694: Epoch   0 Batch 4975/28124   train_loss = 0.476  accuracy = 0.8125  auc = 0.7371428571428571\n",
      "2017-12-30T20:27:43.477450: Epoch   0 Batch 5000/28124   train_loss = 0.480  accuracy = 0.8125  auc = 0.765625\n",
      "2017-12-30T20:27:44.742624: Epoch   0 Batch 5025/28124   train_loss = 0.478  accuracy = 0.78125  auc = 0.7083333333333334\n",
      "2017-12-30T20:27:46.007501: Epoch   0 Batch 5050/28124   train_loss = 0.521  accuracy = 0.8125  auc = 0.8250000000000001\n",
      "2017-12-30T20:27:47.274251: Epoch   0 Batch 5075/28124   train_loss = 0.483  accuracy = 0.75  auc = 0.8212560386473431\n",
      "2017-12-30T20:27:48.540668: Epoch   0 Batch 5100/28124   train_loss = 0.550  accuracy = 0.71875  auc = 0.8225108225108225\n",
      "2017-12-30T20:27:49.784105: Epoch   0 Batch 5125/28124   train_loss = 0.659  accuracy = 0.65625  auc = 0.6363636363636365\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:27:51.083502: Epoch   0 Batch 5150/28124   train_loss = 0.546  accuracy = 0.6875  auc = 0.7705627705627706\n",
      "2017-12-30T20:27:52.334583: Epoch   0 Batch 5175/28124   train_loss = 0.255  accuracy = 0.90625  auc = 0.896551724137931\n",
      "2017-12-30T20:27:53.569724: Epoch   0 Batch 5200/28124   train_loss = 0.524  accuracy = 0.78125  auc = 0.6822916666666666\n",
      "2017-12-30T20:27:54.803100: Epoch   0 Batch 5225/28124   train_loss = 0.387  accuracy = 0.8125  auc = 0.8461538461538461\n",
      "2017-12-30T20:27:56.099476: Epoch   0 Batch 5250/28124   train_loss = 0.346  accuracy = 0.8125  auc = 0.9038461538461537\n",
      "2017-12-30T20:27:57.326983: Epoch   0 Batch 5275/28124   train_loss = 0.531  accuracy = 0.78125  auc = 0.5961538461538461\n",
      "2017-12-30T20:27:58.546588: Epoch   0 Batch 5300/28124   train_loss = 0.531  accuracy = 0.75  auc = 0.7954545454545455\n",
      "2017-12-30T20:27:59.764134: Epoch   0 Batch 5325/28124   train_loss = 0.416  accuracy = 0.75  auc = 0.9454545454545453\n",
      "2017-12-30T20:28:01.044222: Epoch   0 Batch 5350/28124   train_loss = 0.605  accuracy = 0.6875  auc = 0.6818181818181819\n",
      "2017-12-30T20:28:02.286791: Epoch   0 Batch 5375/28124   train_loss = 0.462  accuracy = 0.75  auc = 0.8954545454545455\n",
      "2017-12-30T20:28:03.504221: Epoch   0 Batch 5400/28124   train_loss = 0.485  accuracy = 0.71875  auc = 0.909090909090909\n",
      "2017-12-30T20:28:04.720072: Epoch   0 Batch 5425/28124   train_loss = 0.631  accuracy = 0.71875  auc = 0.5845410628019323\n",
      "2017-12-30T20:28:05.946185: Epoch   0 Batch 5450/28124   train_loss = 0.495  accuracy = 0.78125  auc = 0.7729468599033816\n",
      "2017-12-30T20:28:07.206959: Epoch   0 Batch 5475/28124   train_loss = 0.363  accuracy = 0.90625  auc = 0.8397435897435898\n",
      "2017-12-30T20:28:08.423963: Epoch   0 Batch 5500/28124   train_loss = 0.510  accuracy = 0.71875  auc = 0.7863636363636364\n",
      "2017-12-30T20:28:09.636523: Epoch   0 Batch 5525/28124   train_loss = 0.399  accuracy = 0.8125  auc = 0.8742857142857143\n",
      "2017-12-30T20:28:10.880689: Epoch   0 Batch 5550/28124   train_loss = 0.498  accuracy = 0.8125  auc = 0.6875\n",
      "2017-12-30T20:28:12.131077: Epoch   0 Batch 5575/28124   train_loss = 0.311  accuracy = 0.875  auc = 0.9481481481481482\n",
      "2017-12-30T20:28:13.347505: Epoch   0 Batch 5600/28124   train_loss = 0.411  accuracy = 0.84375  auc = 0.7407407407407407\n",
      "2017-12-30T20:28:14.582069: Epoch   0 Batch 5625/28124   train_loss = 0.448  accuracy = 0.84375  auc = 0.826086956521739\n",
      "2017-12-30T20:28:15.776328: Epoch   0 Batch 5650/28124   train_loss = 0.354  accuracy = 0.90625  auc = 0.7999999999999999\n",
      "2017-12-30T20:28:16.999204: Epoch   0 Batch 5675/28124   train_loss = 0.549  accuracy = 0.75  auc = 0.7727272727272728\n",
      "2017-12-30T20:28:18.253670: Epoch   0 Batch 5700/28124   train_loss = 0.465  accuracy = 0.71875  auc = 0.8744588744588745\n",
      "2017-12-30T20:28:19.455322: Epoch   0 Batch 5725/28124   train_loss = 0.664  accuracy = 0.625  auc = 0.8218623481781376\n",
      "2017-12-30T20:28:20.651356: Epoch   0 Batch 5750/28124   train_loss = 0.414  accuracy = 0.78125  auc = 0.9033816425120773\n",
      "2017-12-30T20:28:21.858652: Epoch   0 Batch 5775/28124   train_loss = 0.366  accuracy = 0.875  auc = 0.7931034482758621\n",
      "2017-12-30T20:28:23.091511: Epoch   0 Batch 5800/28124   train_loss = 0.341  accuracy = 0.84375  auc = 0.8392857142857142\n",
      "2017-12-30T20:28:24.306569: Epoch   0 Batch 5825/28124   train_loss = 0.615  accuracy = 0.625  auc = 0.7166666666666667\n",
      "2017-12-30T20:28:25.540322: Epoch   0 Batch 5850/28124   train_loss = 0.323  accuracy = 0.875  auc = 0.923076923076923\n",
      "2017-12-30T20:28:26.742953: Epoch   0 Batch 5875/28124   train_loss = 0.411  accuracy = 0.875  auc = 0.6160714285714286\n",
      "2017-12-30T20:28:27.968810: Epoch   0 Batch 5900/28124   train_loss = 0.564  accuracy = 0.75  auc = 0.671875\n",
      "2017-12-30T20:28:29.200298: Epoch   0 Batch 5925/28124   train_loss = 0.399  accuracy = 0.84375  auc = 0.8057142857142856\n",
      "2017-12-30T20:28:30.413978: Epoch   0 Batch 5950/28124   train_loss = 0.543  accuracy = 0.71875  auc = 0.7835497835497836\n",
      "2017-12-30T20:28:31.625746: Epoch   0 Batch 5975/28124   train_loss = 0.536  accuracy = 0.71875  auc = 0.7083333333333334\n",
      "2017-12-30T20:28:32.809432: Epoch   0 Batch 6000/28124   train_loss = 0.564  accuracy = 0.6875  auc = 0.681159420289855\n",
      "2017-12-30T20:28:34.045264: Epoch   0 Batch 6025/28124   train_loss = 0.485  accuracy = 0.78125  auc = 0.8318181818181818\n",
      "2017-12-30T20:28:35.265340: Epoch   0 Batch 6050/28124   train_loss = 0.462  accuracy = 0.78125  auc = 0.7916666666666666\n",
      "2017-12-30T20:28:36.478051: Epoch   0 Batch 6075/28124   train_loss = 0.388  accuracy = 0.78125  auc = 0.9613526570048309\n",
      "2017-12-30T20:28:37.687282: Epoch   0 Batch 6100/28124   train_loss = 0.422  accuracy = 0.78125  auc = 0.7756410256410255\n",
      "2017-12-30T20:28:38.909817: Epoch   0 Batch 6125/28124   train_loss = 0.580  accuracy = 0.78125  auc = 0.6811594202898551\n",
      "2017-12-30T20:28:40.119967: Epoch   0 Batch 6150/28124   train_loss = 0.409  accuracy = 0.84375  auc = 0.762962962962963\n",
      "2017-12-30T20:28:41.311860: Epoch   0 Batch 6175/28124   train_loss = 0.496  accuracy = 0.78125  auc = 0.8227272727272728\n",
      "2017-12-30T20:28:42.512085: Epoch   0 Batch 6200/28124   train_loss = 0.290  accuracy = 0.90625  auc = 0.8974358974358975\n",
      "2017-12-30T20:28:43.697676: Epoch   0 Batch 6225/28124   train_loss = 0.622  accuracy = 0.65625  auc = 0.6727272727272728\n",
      "2017-12-30T20:28:44.911460: Epoch   0 Batch 6250/28124   train_loss = 0.500  accuracy = 0.78125  auc = 0.7835497835497836\n",
      "2017-12-30T20:28:46.156909: Epoch   0 Batch 6275/28124   train_loss = 0.503  accuracy = 0.78125  auc = 0.6514285714285715\n",
      "2017-12-30T20:28:47.334911: Epoch   0 Batch 6300/28124   train_loss = 0.369  accuracy = 0.84375  auc = 0.890625\n",
      "2017-12-30T20:28:48.520110: Epoch   0 Batch 6325/28124   train_loss = 0.425  accuracy = 0.78125  auc = 0.8057142857142857\n",
      "2017-12-30T20:28:49.709818: Epoch   0 Batch 6350/28124   train_loss = 0.425  accuracy = 0.78125  auc = 0.7333333333333334\n",
      "2017-12-30T20:28:50.958875: Epoch   0 Batch 6375/28124   train_loss = 0.575  accuracy = 0.75  auc = 0.7454545454545455\n",
      "2017-12-30T20:28:52.219603: Epoch   0 Batch 6400/28124   train_loss = 0.404  accuracy = 0.75  auc = 0.8888888888888888\n",
      "2017-12-30T20:28:53.411982: Epoch   0 Batch 6425/28124   train_loss = 0.379  accuracy = 0.78125  auc = 0.8800000000000001\n",
      "2017-12-30T20:28:54.653593: Epoch   0 Batch 6450/28124   train_loss = 0.266  accuracy = 0.84375  auc = 0.9555555555555555\n",
      "2017-12-30T20:28:55.869984: Epoch   0 Batch 6475/28124   train_loss = 0.614  accuracy = 0.71875  auc = 0.7625000000000001\n",
      "2017-12-30T20:28:57.098264: Epoch   0 Batch 6500/28124   train_loss = 0.474  accuracy = 0.78125  auc = 0.7142857142857143\n",
      "2017-12-30T20:28:58.304802: Epoch   0 Batch 6525/28124   train_loss = 0.391  accuracy = 0.78125  auc = 0.8742857142857143\n",
      "2017-12-30T20:28:59.486572: Epoch   0 Batch 6550/28124   train_loss = 0.509  accuracy = 0.71875  auc = 0.8227272727272728\n",
      "2017-12-30T20:29:00.668698: Epoch   0 Batch 6575/28124   train_loss = 0.447  accuracy = 0.84375  auc = 0.5172413793103449\n",
      "2017-12-30T20:29:01.910030: Epoch   0 Batch 6600/28124   train_loss = 0.654  accuracy = 0.75  auc = 0.6363636363636365\n",
      "2017-12-30T20:29:03.508063: Epoch   0 Batch 6625/28124   train_loss = 0.726  accuracy = 0.59375  auc = 0.7333333333333334\n",
      "2017-12-30T20:29:05.356692: Epoch   0 Batch 6650/28124   train_loss = 0.388  accuracy = 0.8125  auc = 0.7333333333333334\n",
      "2017-12-30T20:29:06.916492: Epoch   0 Batch 6675/28124   train_loss = 0.477  accuracy = 0.8125  auc = 0.5037037037037038\n",
      "2017-12-30T20:29:08.768059: Epoch   0 Batch 6700/28124   train_loss = 0.444  accuracy = 0.78125  auc = 0.7714285714285714\n",
      "2017-12-30T20:29:10.588881: Epoch   0 Batch 6725/28124   train_loss = 0.394  accuracy = 0.84375  auc = 0.8645833333333334\n",
      "2017-12-30T20:29:12.215151: Epoch   0 Batch 6750/28124   train_loss = 0.428  accuracy = 0.84375  auc = 0.7435897435897436\n",
      "2017-12-30T20:29:13.795136: Epoch   0 Batch 6775/28124   train_loss = 0.342  accuracy = 0.90625  auc = 0.8148148148148149\n",
      "2017-12-30T20:29:15.199068: Epoch   0 Batch 6800/28124   train_loss = 0.510  accuracy = 0.71875  auc = 0.8363636363636364\n",
      "2017-12-30T20:29:16.441415: Epoch   0 Batch 6825/28124   train_loss = 0.321  accuracy = 0.8125  auc = 0.9294871794871794\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:29:17.629025: Epoch   0 Batch 6850/28124   train_loss = 0.455  accuracy = 0.8125  auc = 0.8571428571428572\n",
      "2017-12-30T20:29:18.828030: Epoch   0 Batch 6875/28124   train_loss = 0.500  accuracy = 0.8125  auc = 0.7291666666666667\n",
      "2017-12-30T20:29:20.023635: Epoch   0 Batch 6900/28124   train_loss = 0.396  accuracy = 0.875  auc = 0.8285714285714285\n",
      "2017-12-30T20:29:21.395318: Epoch   0 Batch 6925/28124   train_loss = 0.626  accuracy = 0.6875  auc = 0.7291666666666667\n",
      "2017-12-30T20:29:22.628247: Epoch   0 Batch 6950/28124   train_loss = 0.470  accuracy = 0.78125  auc = 0.8309178743961352\n",
      "2017-12-30T20:29:23.787559: Epoch   0 Batch 6975/28124   train_loss = 0.461  accuracy = 0.8125  auc = 0.8260869565217391\n",
      "2017-12-30T20:29:25.288440: Epoch   0 Batch 7000/28124   train_loss = 0.461  accuracy = 0.8125  auc = 0.7922705314009661\n",
      "2017-12-30T20:29:26.739459: Epoch   0 Batch 7025/28124   train_loss = 0.445  accuracy = 0.8125  auc = 0.8072916666666666\n",
      "2017-12-30T20:29:28.156964: Epoch   0 Batch 7050/28124   train_loss = 0.379  accuracy = 0.84375  auc = 0.8012820512820513\n",
      "2017-12-30T20:29:29.717974: Epoch   0 Batch 7075/28124   train_loss = 0.437  accuracy = 0.8125  auc = 0.935064935064935\n",
      "2017-12-30T20:29:31.909449: Epoch   0 Batch 7100/28124   train_loss = 0.556  accuracy = 0.78125  auc = 0.5771428571428571\n",
      "2017-12-30T20:29:33.872146: Epoch   0 Batch 7125/28124   train_loss = 0.510  accuracy = 0.75  auc = 0.78125\n",
      "2017-12-30T20:29:35.678011: Epoch   0 Batch 7150/28124   train_loss = 0.377  accuracy = 0.78125  auc = 0.8269230769230769\n",
      "2017-12-30T20:29:37.347188: Epoch   0 Batch 7175/28124   train_loss = 0.275  accuracy = 0.9375  auc = 0.9358974358974359\n",
      "2017-12-30T20:29:39.404734: Epoch   0 Batch 7200/28124   train_loss = 0.348  accuracy = 0.875  auc = 0.8717948717948718\n",
      "2017-12-30T20:29:40.942852: Epoch   0 Batch 7225/28124   train_loss = 0.358  accuracy = 0.9375  auc = 0.7053571428571429\n",
      "2017-12-30T20:29:42.217262: Epoch   0 Batch 7250/28124   train_loss = 0.434  accuracy = 0.8125  auc = 0.7307692307692308\n",
      "2017-12-30T20:29:43.645818: Epoch   0 Batch 7275/28124   train_loss = 0.383  accuracy = 0.78125  auc = 0.8685714285714285\n",
      "2017-12-30T20:29:45.121061: Epoch   0 Batch 7300/28124   train_loss = 0.549  accuracy = 0.71875  auc = 0.8125\n",
      "2017-12-30T20:29:46.299686: Epoch   0 Batch 7325/28124   train_loss = 0.341  accuracy = 0.875  auc = 0.8148148148148148\n",
      "2017-12-30T20:29:47.422337: Epoch   0 Batch 7350/28124   train_loss = 0.473  accuracy = 0.75  auc = 0.6666666666666667\n",
      "2017-12-30T20:29:48.646419: Epoch   0 Batch 7375/28124   train_loss = 0.552  accuracy = 0.75  auc = 0.65625\n",
      "2017-12-30T20:29:49.861390: Epoch   0 Batch 7400/28124   train_loss = 0.615  accuracy = 0.71875  auc = 0.6280193236714976\n",
      "2017-12-30T20:29:50.989663: Epoch   0 Batch 7425/28124   train_loss = 0.331  accuracy = 0.875  auc = 0.8592592592592593\n",
      "2017-12-30T20:29:52.139220: Epoch   0 Batch 7450/28124   train_loss = 0.423  accuracy = 0.75  auc = 0.8072916666666666\n",
      "2017-12-30T20:29:53.254806: Epoch   0 Batch 7475/28124   train_loss = 0.495  accuracy = 0.8125  auc = 0.748792270531401\n",
      "2017-12-30T20:29:54.370522: Epoch   0 Batch 7500/28124   train_loss = 0.413  accuracy = 0.78125  auc = 0.8854166666666666\n",
      "2017-12-30T20:29:55.493917: Epoch   0 Batch 7525/28124   train_loss = 0.377  accuracy = 0.90625  auc = 0.7851851851851852\n",
      "2017-12-30T20:29:56.636832: Epoch   0 Batch 7550/28124   train_loss = 0.422  accuracy = 0.8125  auc = 0.8541666666666667\n",
      "2017-12-30T20:29:57.756171: Epoch   0 Batch 7575/28124   train_loss = 0.331  accuracy = 0.875  auc = 0.8910256410256411\n",
      "2017-12-30T20:29:58.883276: Epoch   0 Batch 7600/28124   train_loss = 0.328  accuracy = 0.90625  auc = 0.875\n",
      "2017-12-30T20:30:00.004118: Epoch   0 Batch 7625/28124   train_loss = 0.441  accuracy = 0.78125  auc = 0.859090909090909\n",
      "2017-12-30T20:30:01.175257: Epoch   0 Batch 7650/28124   train_loss = 0.454  accuracy = 0.75  auc = 0.8405797101449275\n",
      "2017-12-30T20:30:02.288857: Epoch   0 Batch 7675/28124   train_loss = 0.615  accuracy = 0.65625  auc = 0.7791666666666667\n",
      "2017-12-30T20:30:03.387880: Epoch   0 Batch 7700/28124   train_loss = 0.537  accuracy = 0.71875  auc = 0.6354166666666666\n",
      "2017-12-30T20:30:04.506490: Epoch   0 Batch 7725/28124   train_loss = 0.678  accuracy = 0.5625  auc = 0.80078125\n",
      "2017-12-30T20:30:05.622636: Epoch   0 Batch 7750/28124   train_loss = 0.471  accuracy = 0.75  auc = 0.796875\n",
      "2017-12-30T20:30:06.778495: Epoch   0 Batch 7775/28124   train_loss = 0.466  accuracy = 0.78125  auc = 0.796875\n",
      "2017-12-30T20:30:08.273262: Epoch   0 Batch 7800/28124   train_loss = 0.312  accuracy = 0.8125  auc = 0.9542857142857143\n",
      "2017-12-30T20:30:09.650434: Epoch   0 Batch 7825/28124   train_loss = 0.386  accuracy = 0.875  auc = 0.7185185185185186\n",
      "2017-12-30T20:30:10.886440: Epoch   0 Batch 7850/28124   train_loss = 0.352  accuracy = 0.875  auc = 0.8461538461538461\n",
      "2017-12-30T20:30:12.527881: Epoch   0 Batch 7875/28124   train_loss = 0.444  accuracy = 0.8125  auc = 0.7371794871794871\n",
      "2017-12-30T20:30:13.703494: Epoch   0 Batch 7900/28124   train_loss = 0.602  accuracy = 0.6875  auc = 0.6909090909090909\n",
      "2017-12-30T20:30:14.821128: Epoch   0 Batch 7925/28124   train_loss = 0.390  accuracy = 0.8125  auc = 0.90625\n",
      "2017-12-30T20:30:15.967870: Epoch   0 Batch 7950/28124   train_loss = 0.532  accuracy = 0.78125  auc = 0.6770833333333334\n",
      "2017-12-30T20:30:17.141661: Epoch   0 Batch 7975/28124   train_loss = 0.419  accuracy = 0.78125  auc = 0.8697916666666667\n",
      "2017-12-30T20:30:18.272707: Epoch   0 Batch 8000/28124   train_loss = 0.363  accuracy = 0.90625  auc = 0.6339285714285714\n",
      "2017-12-30T20:30:19.383943: Epoch   0 Batch 8025/28124   train_loss = 0.484  accuracy = 0.75  auc = 0.8917748917748918\n",
      "2017-12-30T20:30:20.498517: Epoch   0 Batch 8050/28124   train_loss = 0.513  accuracy = 0.8125  auc = 0.6285714285714286\n",
      "2017-12-30T20:30:21.637537: Epoch   0 Batch 8075/28124   train_loss = 0.362  accuracy = 0.875  auc = 0.9567099567099567\n",
      "2017-12-30T20:30:22.749101: Epoch   0 Batch 8100/28124   train_loss = 0.547  accuracy = 0.71875  auc = 0.7625\n",
      "2017-12-30T20:30:23.886214: Epoch   0 Batch 8125/28124   train_loss = 0.523  accuracy = 0.75  auc = 0.8405797101449275\n",
      "2017-12-30T20:30:25.012805: Epoch   0 Batch 8150/28124   train_loss = 0.425  accuracy = 0.78125  auc = 0.8385416666666667\n",
      "2017-12-30T20:30:26.198696: Epoch   0 Batch 8175/28124   train_loss = 0.365  accuracy = 0.875  auc = 0.8958333333333333\n",
      "2017-12-30T20:30:27.321651: Epoch   0 Batch 8200/28124   train_loss = 0.385  accuracy = 0.8125  auc = 0.8914285714285715\n",
      "2017-12-30T20:30:28.468468: Epoch   0 Batch 8225/28124   train_loss = 0.563  accuracy = 0.71875  auc = 0.91015625\n",
      "2017-12-30T20:30:29.604860: Epoch   0 Batch 8250/28124   train_loss = 0.350  accuracy = 0.84375  auc = 0.8782051282051282\n",
      "2017-12-30T20:30:30.742518: Epoch   0 Batch 8275/28124   train_loss = 0.516  accuracy = 0.75  auc = 0.7922077922077921\n",
      "2017-12-30T20:30:31.888159: Epoch   0 Batch 8300/28124   train_loss = 0.413  accuracy = 0.8125  auc = 0.8333333333333334\n",
      "2017-12-30T20:30:33.035928: Epoch   0 Batch 8325/28124   train_loss = 0.324  accuracy = 0.84375  auc = 0.8482142857142856\n",
      "2017-12-30T20:30:34.212595: Epoch   0 Batch 8350/28124   train_loss = 0.447  accuracy = 0.84375  auc = 0.75\n",
      "2017-12-30T20:30:35.353212: Epoch   0 Batch 8375/28124   train_loss = 0.313  accuracy = 0.875  auc = 0.9375\n",
      "2017-12-30T20:30:36.481679: Epoch   0 Batch 8400/28124   train_loss = 0.617  accuracy = 0.6875  auc = 0.7583333333333333\n",
      "2017-12-30T20:30:37.630156: Epoch   0 Batch 8425/28124   train_loss = 0.433  accuracy = 0.8125  auc = 0.7257142857142858\n",
      "2017-12-30T20:30:38.750581: Epoch   0 Batch 8450/28124   train_loss = 0.518  accuracy = 0.78125  auc = 0.6742857142857143\n",
      "2017-12-30T20:30:39.884018: Epoch   0 Batch 8475/28124   train_loss = 0.401  accuracy = 0.8125  auc = 0.7678571428571428\n",
      "2017-12-30T20:30:41.044240: Epoch   0 Batch 8500/28124   train_loss = 0.478  accuracy = 0.78125  auc = 0.765625\n",
      "2017-12-30T20:30:42.205187: Epoch   0 Batch 8525/28124   train_loss = 0.287  accuracy = 0.875  auc = 0.9107142857142857\n",
      "2017-12-30T20:30:43.325616: Epoch   0 Batch 8550/28124   train_loss = 0.321  accuracy = 0.90625  auc = 0.896551724137931\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:30:44.451031: Epoch   0 Batch 8575/28124   train_loss = 0.384  accuracy = 0.84375  auc = 0.8457142857142858\n",
      "2017-12-30T20:30:45.604457: Epoch   0 Batch 8600/28124   train_loss = 0.439  accuracy = 0.75  auc = 0.8454106280193237\n",
      "2017-12-30T20:30:46.726684: Epoch   0 Batch 8625/28124   train_loss = 0.494  accuracy = 0.65625  auc = 0.9352226720647774\n",
      "2017-12-30T20:30:47.873826: Epoch   0 Batch 8650/28124   train_loss = 0.350  accuracy = 0.8125  auc = 0.953125\n",
      "2017-12-30T20:30:48.999030: Epoch   0 Batch 8675/28124   train_loss = 0.426  accuracy = 0.84375  auc = 0.7243589743589743\n",
      "2017-12-30T20:30:50.172991: Epoch   0 Batch 8700/28124   train_loss = 0.388  accuracy = 0.8125  auc = 0.8840579710144928\n",
      "2017-12-30T20:30:51.354534: Epoch   0 Batch 8725/28124   train_loss = 0.568  accuracy = 0.6875  auc = 0.7246376811594203\n",
      "2017-12-30T20:30:52.484380: Epoch   0 Batch 8750/28124   train_loss = 0.360  accuracy = 0.8125  auc = 0.9166666666666666\n",
      "2017-12-30T20:30:53.648349: Epoch   0 Batch 8775/28124   train_loss = 0.564  accuracy = 0.75  auc = 0.578125\n",
      "2017-12-30T20:30:54.784959: Epoch   0 Batch 8800/28124   train_loss = 0.542  accuracy = 0.6875  auc = 0.7662337662337664\n",
      "2017-12-30T20:30:55.944313: Epoch   0 Batch 8825/28124   train_loss = 0.288  accuracy = 0.90625  auc = 0.9555555555555556\n",
      "2017-12-30T20:30:57.124841: Epoch   0 Batch 8850/28124   train_loss = 0.455  accuracy = 0.8125  auc = 0.7599999999999999\n",
      "2017-12-30T20:30:58.261136: Epoch   0 Batch 8875/28124   train_loss = 0.577  accuracy = 0.71875  auc = 0.6666666666666667\n",
      "2017-12-30T20:30:59.384923: Epoch   0 Batch 8900/28124   train_loss = 0.486  accuracy = 0.8125  auc = 0.6571428571428571\n",
      "2017-12-30T20:31:00.517878: Epoch   0 Batch 8925/28124   train_loss = 0.591  accuracy = 0.6875  auc = 0.735930735930736\n",
      "2017-12-30T20:31:01.656008: Epoch   0 Batch 8950/28124   train_loss = 0.440  accuracy = 0.8125  auc = 0.8961038961038962\n",
      "2017-12-30T20:31:02.802809: Epoch   0 Batch 8975/28124   train_loss = 0.565  accuracy = 0.75  auc = 0.7575757575757576\n",
      "2017-12-30T20:31:03.931075: Epoch   0 Batch 9000/28124   train_loss = 0.383  accuracy = 0.84375  auc = 0.8269230769230769\n",
      "2017-12-30T20:31:05.103114: Epoch   0 Batch 9025/28124   train_loss = 0.374  accuracy = 0.8125  auc = 0.75\n",
      "2017-12-30T20:31:06.249936: Epoch   0 Batch 9050/28124   train_loss = 0.358  accuracy = 0.84375  auc = 0.8628571428571428\n",
      "2017-12-30T20:31:07.381341: Epoch   0 Batch 9075/28124   train_loss = 0.377  accuracy = 0.875  auc = 0.7564102564102564\n",
      "2017-12-30T20:31:08.516796: Epoch   0 Batch 9100/28124   train_loss = 0.495  accuracy = 0.71875  auc = 0.8484848484848486\n",
      "2017-12-30T20:31:09.644877: Epoch   0 Batch 9125/28124   train_loss = 0.495  accuracy = 0.75  auc = 0.6971428571428571\n",
      "2017-12-30T20:31:10.767987: Epoch   0 Batch 9150/28124   train_loss = 0.420  accuracy = 0.84375  auc = 0.8409090909090909\n",
      "2017-12-30T20:31:11.928133: Epoch   0 Batch 9175/28124   train_loss = 0.424  accuracy = 0.75  auc = 0.8695652173913044\n",
      "2017-12-30T20:31:13.093059: Epoch   0 Batch 9200/28124   train_loss = 0.540  accuracy = 0.75  auc = 0.671875\n",
      "2017-12-30T20:31:14.227045: Epoch   0 Batch 9225/28124   train_loss = 0.432  accuracy = 0.75  auc = 0.8743961352657004\n",
      "2017-12-30T20:31:15.351502: Epoch   0 Batch 9250/28124   train_loss = 0.528  accuracy = 0.71875  auc = 0.7198067632850241\n",
      "2017-12-30T20:31:16.485508: Epoch   0 Batch 9275/28124   train_loss = 0.436  accuracy = 0.78125  auc = 0.796875\n",
      "2017-12-30T20:31:17.634609: Epoch   0 Batch 9300/28124   train_loss = 0.469  accuracy = 0.71875  auc = 0.7435897435897436\n",
      "2017-12-30T20:31:18.781302: Epoch   0 Batch 9325/28124   train_loss = 0.434  accuracy = 0.8125  auc = 0.8787878787878788\n",
      "2017-12-30T20:31:19.958914: Epoch   0 Batch 9350/28124   train_loss = 0.428  accuracy = 0.75  auc = 0.8171428571428572\n",
      "2017-12-30T20:31:21.280717: Epoch   0 Batch 9375/28124   train_loss = 0.713  accuracy = 0.53125  auc = 0.7803921568627451\n",
      "2017-12-30T20:31:22.454968: Epoch   0 Batch 9400/28124   train_loss = 0.577  accuracy = 0.71875  auc = 0.6956521739130436\n",
      "2017-12-30T20:31:23.678318: Epoch   0 Batch 9425/28124   train_loss = 0.378  accuracy = 0.78125  auc = 0.9314285714285715\n",
      "2017-12-30T20:31:24.807639: Epoch   0 Batch 9450/28124   train_loss = 0.423  accuracy = 0.78125  auc = 0.8114285714285715\n",
      "2017-12-30T20:31:25.949324: Epoch   0 Batch 9475/28124   train_loss = 0.535  accuracy = 0.71875  auc = 0.7391304347826086\n",
      "2017-12-30T20:31:27.133118: Epoch   0 Batch 9500/28124   train_loss = 0.488  accuracy = 0.75  auc = 0.7971014492753623\n",
      "2017-12-30T20:31:28.259672: Epoch   0 Batch 9525/28124   train_loss = 0.608  accuracy = 0.6875  auc = 0.6681818181818181\n",
      "2017-12-30T20:31:29.424756: Epoch   0 Batch 9550/28124   train_loss = 0.441  accuracy = 0.8125  auc = 0.7942857142857143\n",
      "2017-12-30T20:31:30.588098: Epoch   0 Batch 9575/28124   train_loss = 0.373  accuracy = 0.8125  auc = 0.8160919540229885\n",
      "2017-12-30T20:31:31.764697: Epoch   0 Batch 9600/28124   train_loss = 0.484  accuracy = 0.8125  auc = 0.6628571428571428\n",
      "2017-12-30T20:31:32.961198: Epoch   0 Batch 9625/28124   train_loss = 0.229  accuracy = 0.9375  auc = 0.9642857142857143\n",
      "2017-12-30T20:31:34.151601: Epoch   0 Batch 9650/28124   train_loss = 0.357  accuracy = 0.875  auc = 0.9028571428571428\n",
      "2017-12-30T20:31:35.295908: Epoch   0 Batch 9675/28124   train_loss = 0.537  accuracy = 0.75  auc = 0.625\n",
      "2017-12-30T20:31:36.421206: Epoch   0 Batch 9700/28124   train_loss = 0.483  accuracy = 0.75  auc = 0.8136363636363636\n",
      "2017-12-30T20:31:37.595484: Epoch   0 Batch 9725/28124   train_loss = 0.477  accuracy = 0.8125  auc = 0.75\n",
      "2017-12-30T20:31:38.734183: Epoch   0 Batch 9750/28124   train_loss = 0.574  accuracy = 0.75  auc = 0.6285714285714286\n",
      "2017-12-30T20:31:39.882221: Epoch   0 Batch 9775/28124   train_loss = 0.306  accuracy = 0.84375  auc = 0.875\n",
      "2017-12-30T20:31:41.044315: Epoch   0 Batch 9800/28124   train_loss = 0.515  accuracy = 0.71875  auc = 0.8208333333333334\n",
      "2017-12-30T20:31:42.204948: Epoch   0 Batch 9825/28124   train_loss = 0.466  accuracy = 0.75  auc = 0.8500000000000001\n",
      "2017-12-30T20:31:43.351818: Epoch   0 Batch 9850/28124   train_loss = 0.403  accuracy = 0.84375  auc = 0.8647342995169082\n",
      "2017-12-30T20:31:44.478371: Epoch   0 Batch 9875/28124   train_loss = 0.528  accuracy = 0.6875  auc = 0.855072463768116\n",
      "2017-12-30T20:31:45.625207: Epoch   0 Batch 9900/28124   train_loss = 0.466  accuracy = 0.78125  auc = 0.8623481781376517\n",
      "2017-12-30T20:31:46.767857: Epoch   0 Batch 9925/28124   train_loss = 0.412  accuracy = 0.8125  auc = 0.9\n",
      "2017-12-30T20:31:47.892207: Epoch   0 Batch 9950/28124   train_loss = 0.431  accuracy = 0.78125  auc = 0.8171428571428572\n",
      "2017-12-30T20:31:49.050975: Epoch   0 Batch 9975/28124   train_loss = 0.510  accuracy = 0.75  auc = 0.8441558441558441\n",
      "2017-12-30T20:31:50.213763: Epoch   0 Batch 10000/28124   train_loss = 0.507  accuracy = 0.75  auc = 0.8067632850241546\n",
      "2017-12-30T20:31:51.339639: Epoch   0 Batch 10025/28124   train_loss = 0.619  accuracy = 0.71875  auc = 0.6875\n",
      "2017-12-30T20:31:52.479661: Epoch   0 Batch 10050/28124   train_loss = 0.475  accuracy = 0.78125  auc = 0.8318181818181818\n",
      "2017-12-30T20:31:53.615576: Epoch   0 Batch 10075/28124   train_loss = 0.437  accuracy = 0.8125  auc = 0.8744588744588744\n",
      "2017-12-30T20:31:54.761192: Epoch   0 Batch 10100/28124   train_loss = 0.633  accuracy = 0.71875  auc = 0.42857142857142855\n",
      "2017-12-30T20:31:55.931920: Epoch   0 Batch 10125/28124   train_loss = 0.461  accuracy = 0.75  auc = 0.7916666666666667\n",
      "2017-12-30T20:31:57.070441: Epoch   0 Batch 10150/28124   train_loss = 0.565  accuracy = 0.75  auc = 0.5657142857142857\n",
      "2017-12-30T20:31:58.213079: Epoch   0 Batch 10175/28124   train_loss = 0.557  accuracy = 0.75  auc = 0.6666666666666667\n",
      "2017-12-30T20:31:59.337423: Epoch   0 Batch 10200/28124   train_loss = 0.388  accuracy = 0.84375  auc = 0.84\n",
      "2017-12-30T20:32:00.480139: Epoch   0 Batch 10225/28124   train_loss = 0.521  accuracy = 0.78125  auc = 0.7439613526570048\n",
      "2017-12-30T20:32:01.602045: Epoch   0 Batch 10250/28124   train_loss = 0.537  accuracy = 0.6875  auc = 0.7053140096618358\n",
      "2017-12-30T20:32:02.765277: Epoch   0 Batch 10275/28124   train_loss = 0.293  accuracy = 0.875  auc = 0.9310344827586208\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:32:03.892259: Epoch   0 Batch 10300/28124   train_loss = 0.521  accuracy = 0.75  auc = 0.71875\n",
      "2017-12-30T20:32:05.048975: Epoch   0 Batch 10325/28124   train_loss = 0.468  accuracy = 0.78125  auc = 0.8658008658008658\n",
      "2017-12-30T20:32:06.214858: Epoch   0 Batch 10350/28124   train_loss = 0.531  accuracy = 0.6875  auc = 0.7909090909090909\n",
      "2017-12-30T20:32:07.343831: Epoch   0 Batch 10375/28124   train_loss = 0.594  accuracy = 0.71875  auc = 0.7692307692307693\n",
      "2017-12-30T20:32:08.460423: Epoch   0 Batch 10400/28124   train_loss = 0.450  accuracy = 0.8125  auc = 0.7916666666666667\n",
      "2017-12-30T20:32:09.587850: Epoch   0 Batch 10425/28124   train_loss = 0.469  accuracy = 0.8125  auc = 0.7257142857142856\n",
      "2017-12-30T20:32:10.722288: Epoch   0 Batch 10450/28124   train_loss = 0.401  accuracy = 0.875  auc = 0.7885714285714286\n",
      "2017-12-30T20:32:11.847632: Epoch   0 Batch 10475/28124   train_loss = 0.465  accuracy = 0.84375  auc = 0.7257142857142856\n",
      "2017-12-30T20:32:12.979327: Epoch   0 Batch 10500/28124   train_loss = 0.417  accuracy = 0.78125  auc = 0.8114285714285714\n",
      "2017-12-30T20:32:14.122971: Epoch   0 Batch 10525/28124   train_loss = 0.616  accuracy = 0.6875  auc = 0.6318181818181818\n",
      "2017-12-30T20:32:15.242347: Epoch   0 Batch 10550/28124   train_loss = 0.339  accuracy = 0.84375  auc = 0.8160919540229884\n",
      "2017-12-30T20:32:16.365430: Epoch   0 Batch 10575/28124   train_loss = 0.349  accuracy = 0.84375  auc = 0.8461538461538461\n",
      "2017-12-30T20:32:17.478473: Epoch   0 Batch 10600/28124   train_loss = 0.448  accuracy = 0.78125  auc = 0.8164251207729469\n",
      "2017-12-30T20:32:18.583545: Epoch   0 Batch 10625/28124   train_loss = 0.424  accuracy = 0.8125  auc = 0.6794871794871795\n",
      "2017-12-30T20:32:19.704870: Epoch   0 Batch 10650/28124   train_loss = 0.530  accuracy = 0.75  auc = 0.7342995169082126\n",
      "2017-12-30T20:32:20.830327: Epoch   0 Batch 10675/28124   train_loss = 0.486  accuracy = 0.875  auc = 0.8354978354978354\n",
      "2017-12-30T20:32:21.936057: Epoch   0 Batch 10700/28124   train_loss = 0.375  accuracy = 0.8125  auc = 0.8802083333333333\n",
      "2017-12-30T20:32:23.085499: Epoch   0 Batch 10725/28124   train_loss = 0.308  accuracy = 0.875  auc = 0.9230769230769231\n",
      "2017-12-30T20:32:24.197573: Epoch   0 Batch 10750/28124   train_loss = 0.387  accuracy = 0.84375  auc = 0.8285714285714285\n",
      "2017-12-30T20:32:25.318523: Epoch   0 Batch 10775/28124   train_loss = 0.339  accuracy = 0.84375  auc = 0.7589285714285714\n",
      "2017-12-30T20:32:26.457570: Epoch   0 Batch 10800/28124   train_loss = 0.377  accuracy = 0.84375  auc = 0.8742857142857143\n",
      "2017-12-30T20:32:27.573640: Epoch   0 Batch 10825/28124   train_loss = 0.388  accuracy = 0.8125  auc = 0.8342857142857143\n",
      "2017-12-30T20:32:28.692417: Epoch   0 Batch 10850/28124   train_loss = 0.370  accuracy = 0.84375  auc = 0.90625\n",
      "2017-12-30T20:32:29.795957: Epoch   0 Batch 10875/28124   train_loss = 0.510  accuracy = 0.71875  auc = 0.8\n",
      "2017-12-30T20:32:30.921325: Epoch   0 Batch 10900/28124   train_loss = 0.545  accuracy = 0.78125  auc = 0.7181818181818181\n",
      "2017-12-30T20:32:32.086964: Epoch   0 Batch 10925/28124   train_loss = 0.491  accuracy = 0.6875  auc = 0.894736842105263\n",
      "2017-12-30T20:32:33.199780: Epoch   0 Batch 10950/28124   train_loss = 0.337  accuracy = 0.84375  auc = 0.8666666666666667\n",
      "2017-12-30T20:32:34.338498: Epoch   0 Batch 10975/28124   train_loss = 0.566  accuracy = 0.6875  auc = 0.7833333333333333\n",
      "2017-12-30T20:32:35.452446: Epoch   0 Batch 11000/28124   train_loss = 0.305  accuracy = 0.875  auc = 0.9423076923076923\n",
      "2017-12-30T20:32:36.560024: Epoch   0 Batch 11025/28124   train_loss = 0.441  accuracy = 0.75  auc = 0.8125\n",
      "2017-12-30T20:32:37.709179: Epoch   0 Batch 11050/28124   train_loss = 0.345  accuracy = 0.875  auc = 0.8148148148148149\n",
      "2017-12-30T20:32:38.838082: Epoch   0 Batch 11075/28124   train_loss = 0.367  accuracy = 0.8125  auc = 0.8717948717948718\n",
      "2017-12-30T20:32:39.966964: Epoch   0 Batch 11100/28124   train_loss = 0.542  accuracy = 0.78125  auc = 0.6875\n",
      "2017-12-30T20:32:41.128792: Epoch   0 Batch 11125/28124   train_loss = 0.475  accuracy = 0.78125  auc = 0.765625\n",
      "2017-12-30T20:32:42.245697: Epoch   0 Batch 11150/28124   train_loss = 0.725  accuracy = 0.625  auc = 0.6761133603238867\n",
      "2017-12-30T20:32:43.392135: Epoch   0 Batch 11175/28124   train_loss = 0.490  accuracy = 0.78125  auc = 0.7632850241545894\n",
      "2017-12-30T20:32:44.529895: Epoch   0 Batch 11200/28124   train_loss = 0.547  accuracy = 0.6875  auc = 0.6857142857142857\n",
      "2017-12-30T20:32:45.639168: Epoch   0 Batch 11225/28124   train_loss = 0.525  accuracy = 0.75  auc = 0.748792270531401\n",
      "2017-12-30T20:32:46.772202: Epoch   0 Batch 11250/28124   train_loss = 0.612  accuracy = 0.75  auc = 0.6493506493506493\n",
      "2017-12-30T20:32:47.888215: Epoch   0 Batch 11275/28124   train_loss = 0.331  accuracy = 0.84375  auc = 0.8666666666666667\n",
      "2017-12-30T20:32:49.041886: Epoch   0 Batch 11300/28124   train_loss = 0.373  accuracy = 0.84375  auc = 0.8\n",
      "2017-12-30T20:32:50.219423: Epoch   0 Batch 11325/28124   train_loss = 0.522  accuracy = 0.8125  auc = 0.800865800865801\n",
      "2017-12-30T20:32:51.435347: Epoch   0 Batch 11350/28124   train_loss = 0.338  accuracy = 0.8125  auc = 0.8742857142857143\n",
      "2017-12-30T20:32:52.598513: Epoch   0 Batch 11375/28124   train_loss = 0.551  accuracy = 0.71875  auc = 0.7053140096618358\n",
      "2017-12-30T20:32:53.745922: Epoch   0 Batch 11400/28124   train_loss = 0.319  accuracy = 0.90625  auc = 0.9610389610389611\n",
      "2017-12-30T20:32:54.919748: Epoch   0 Batch 11425/28124   train_loss = 0.562  accuracy = 0.75  auc = 0.7748917748917749\n",
      "2017-12-30T20:32:56.103787: Epoch   0 Batch 11450/28124   train_loss = 0.499  accuracy = 0.8125  auc = 0.8095238095238095\n",
      "2017-12-30T20:32:57.233102: Epoch   0 Batch 11475/28124   train_loss = 0.482  accuracy = 0.78125  auc = 0.6602564102564102\n",
      "2017-12-30T20:32:58.357477: Epoch   0 Batch 11500/28124   train_loss = 0.633  accuracy = 0.71875  auc = 0.7529411764705882\n",
      "2017-12-30T20:32:59.485980: Epoch   0 Batch 11525/28124   train_loss = 0.508  accuracy = 0.75  auc = 0.7818181818181817\n",
      "2017-12-30T20:33:00.612602: Epoch   0 Batch 11550/28124   train_loss = 0.541  accuracy = 0.71875  auc = 0.7445887445887446\n",
      "2017-12-30T20:33:01.732065: Epoch   0 Batch 11575/28124   train_loss = 0.542  accuracy = 0.75  auc = 0.6628571428571429\n",
      "2017-12-30T20:33:02.867479: Epoch   0 Batch 11600/28124   train_loss = 0.402  accuracy = 0.875  auc = 0.7628205128205128\n",
      "2017-12-30T20:33:04.085676: Epoch   0 Batch 11625/28124   train_loss = 0.479  accuracy = 0.75  auc = 0.8090909090909091\n",
      "2017-12-30T20:33:05.216744: Epoch   0 Batch 11650/28124   train_loss = 0.481  accuracy = 0.6875  auc = 0.8354978354978355\n",
      "2017-12-30T20:33:06.337246: Epoch   0 Batch 11675/28124   train_loss = 0.411  accuracy = 0.875  auc = 0.6518518518518519\n",
      "2017-12-30T20:33:07.468605: Epoch   0 Batch 11700/28124   train_loss = 0.413  accuracy = 0.84375  auc = 0.7942857142857143\n",
      "2017-12-30T20:33:08.583297: Epoch   0 Batch 11725/28124   train_loss = 0.522  accuracy = 0.6875  auc = 0.7727272727272727\n",
      "2017-12-30T20:33:09.706165: Epoch   0 Batch 11750/28124   train_loss = 0.472  accuracy = 0.8125  auc = 0.765625\n",
      "2017-12-30T20:33:10.820551: Epoch   0 Batch 11775/28124   train_loss = 0.559  accuracy = 0.71875  auc = 0.7681818181818182\n",
      "2017-12-30T20:33:11.978919: Epoch   0 Batch 11800/28124   train_loss = 0.489  accuracy = 0.75  auc = 0.8181818181818181\n",
      "2017-12-30T20:33:13.154312: Epoch   0 Batch 11825/28124   train_loss = 0.520  accuracy = 0.71875  auc = 0.671875\n",
      "2017-12-30T20:33:14.318226: Epoch   0 Batch 11850/28124   train_loss = 0.452  accuracy = 0.75  auc = 0.8727272727272728\n",
      "2017-12-30T20:33:15.441011: Epoch   0 Batch 11875/28124   train_loss = 0.519  accuracy = 0.78125  auc = 0.7727272727272728\n",
      "2017-12-30T20:33:16.581206: Epoch   0 Batch 11900/28124   train_loss = 0.489  accuracy = 0.78125  auc = 0.7542857142857142\n",
      "2017-12-30T20:33:17.739540: Epoch   0 Batch 11925/28124   train_loss = 0.413  accuracy = 0.84375  auc = 0.8171428571428572\n",
      "2017-12-30T20:33:18.851488: Epoch   0 Batch 11950/28124   train_loss = 0.406  accuracy = 0.78125  auc = 0.84\n",
      "2017-12-30T20:33:19.972684: Epoch   0 Batch 11975/28124   train_loss = 0.479  accuracy = 0.78125  auc = 0.7085714285714285\n",
      "2017-12-30T20:33:21.160987: Epoch   0 Batch 12000/28124   train_loss = 0.514  accuracy = 0.6875  auc = 0.8095238095238095\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:33:22.311255: Epoch   0 Batch 12025/28124   train_loss = 0.337  accuracy = 0.8125  auc = 0.8914285714285713\n",
      "2017-12-30T20:33:23.447830: Epoch   0 Batch 12050/28124   train_loss = 0.430  accuracy = 0.78125  auc = 0.8228571428571428\n",
      "2017-12-30T20:33:24.592267: Epoch   0 Batch 12075/28124   train_loss = 0.397  accuracy = 0.84375  auc = 0.8\n",
      "2017-12-30T20:33:25.720813: Epoch   0 Batch 12100/28124   train_loss = 0.296  accuracy = 0.90625  auc = 0.9038461538461539\n",
      "2017-12-30T20:33:26.873562: Epoch   0 Batch 12125/28124   train_loss = 0.455  accuracy = 0.8125  auc = 0.76\n",
      "2017-12-30T20:33:28.030581: Epoch   0 Batch 12150/28124   train_loss = 0.248  accuracy = 0.90625  auc = 0.9821428571428572\n",
      "2017-12-30T20:33:29.177203: Epoch   0 Batch 12175/28124   train_loss = 0.388  accuracy = 0.875  auc = 0.8057142857142857\n",
      "2017-12-30T20:33:30.313970: Epoch   0 Batch 12200/28124   train_loss = 0.475  accuracy = 0.75  auc = 0.7708333333333333\n",
      "2017-12-30T20:33:31.459253: Epoch   0 Batch 12225/28124   train_loss = 0.331  accuracy = 0.875  auc = 0.8303571428571428\n",
      "2017-12-30T20:33:32.588825: Epoch   0 Batch 12250/28124   train_loss = 0.323  accuracy = 0.875  auc = 0.875\n",
      "2017-12-30T20:33:33.742780: Epoch   0 Batch 12275/28124   train_loss = 0.499  accuracy = 0.875  auc = 0.6614583333333333\n",
      "2017-12-30T20:33:34.874303: Epoch   0 Batch 12300/28124   train_loss = 0.593  accuracy = 0.71875  auc = 0.6863636363636364\n",
      "2017-12-30T20:33:36.009030: Epoch   0 Batch 12325/28124   train_loss = 0.439  accuracy = 0.78125  auc = 0.8072916666666667\n",
      "2017-12-30T20:33:37.150699: Epoch   0 Batch 12350/28124   train_loss = 0.391  accuracy = 0.84375  auc = 0.7884615384615384\n",
      "2017-12-30T20:33:38.287563: Epoch   0 Batch 12375/28124   train_loss = 0.414  accuracy = 0.84375  auc = 0.8385416666666667\n",
      "2017-12-30T20:33:39.401843: Epoch   0 Batch 12400/28124   train_loss = 0.373  accuracy = 0.84375  auc = 0.8171428571428571\n",
      "2017-12-30T20:33:40.528257: Epoch   0 Batch 12425/28124   train_loss = 0.492  accuracy = 0.75  auc = 0.75\n",
      "2017-12-30T20:33:41.667735: Epoch   0 Batch 12450/28124   train_loss = 0.511  accuracy = 0.65625  auc = 0.8268398268398268\n",
      "2017-12-30T20:33:42.794885: Epoch   0 Batch 12475/28124   train_loss = 0.465  accuracy = 0.75  auc = 0.8177083333333333\n",
      "2017-12-30T20:33:43.931181: Epoch   0 Batch 12500/28124   train_loss = 0.503  accuracy = 0.75  auc = 0.8311688311688312\n",
      "2017-12-30T20:33:45.740288: Epoch   0 Batch 12525/28124   train_loss = 0.426  accuracy = 0.8125  auc = 0.8593749999999999\n",
      "2017-12-30T20:33:47.096511: Epoch   0 Batch 12550/28124   train_loss = 0.527  accuracy = 0.75  auc = 0.8268398268398268\n",
      "2017-12-30T20:33:48.228843: Epoch   0 Batch 12575/28124   train_loss = 0.629  accuracy = 0.65625  auc = 0.75\n",
      "2017-12-30T20:33:49.349128: Epoch   0 Batch 12600/28124   train_loss = 0.557  accuracy = 0.71875  auc = 0.5714285714285714\n",
      "2017-12-30T20:33:50.493179: Epoch   0 Batch 12625/28124   train_loss = 0.366  accuracy = 0.875  auc = 0.8971428571428571\n",
      "2017-12-30T20:33:51.626050: Epoch   0 Batch 12650/28124   train_loss = 0.444  accuracy = 0.84375  auc = 0.8917748917748918\n",
      "2017-12-30T20:33:52.752777: Epoch   0 Batch 12675/28124   train_loss = 0.607  accuracy = 0.75  auc = 0.6590909090909091\n",
      "2017-12-30T20:33:53.881727: Epoch   0 Batch 12700/28124   train_loss = 0.556  accuracy = 0.6875  auc = 0.7227272727272727\n",
      "2017-12-30T20:33:55.006290: Epoch   0 Batch 12725/28124   train_loss = 0.385  accuracy = 0.84375  auc = 0.8906249999999999\n",
      "2017-12-30T20:33:56.281824: Epoch   0 Batch 12750/28124   train_loss = 0.441  accuracy = 0.75  auc = 0.828125\n",
      "2017-12-30T20:33:58.294452: Epoch   0 Batch 12775/28124   train_loss = 0.500  accuracy = 0.75  auc = 0.6770833333333334\n",
      "2017-12-30T20:33:59.951329: Epoch   0 Batch 12800/28124   train_loss = 0.397  accuracy = 0.84375  auc = 0.725925925925926\n",
      "2017-12-30T20:34:01.574186: Epoch   0 Batch 12825/28124   train_loss = 0.371  accuracy = 0.78125  auc = 0.8914285714285715\n",
      "2017-12-30T20:34:03.467580: Epoch   0 Batch 12850/28124   train_loss = 0.488  accuracy = 0.6875  auc = 0.8067632850241546\n",
      "2017-12-30T20:34:04.903365: Epoch   0 Batch 12875/28124   train_loss = 0.329  accuracy = 0.90625  auc = 0.8\n",
      "2017-12-30T20:34:06.192671: Epoch   0 Batch 12900/28124   train_loss = 0.478  accuracy = 0.75  auc = 0.7760416666666666\n",
      "2017-12-30T20:34:07.458714: Epoch   0 Batch 12925/28124   train_loss = 0.611  accuracy = 0.65625  auc = 0.8531746031746031\n",
      "2017-12-30T20:34:08.885094: Epoch   0 Batch 12950/28124   train_loss = 0.533  accuracy = 0.78125  auc = 0.6197916666666667\n",
      "2017-12-30T20:34:10.263972: Epoch   0 Batch 12975/28124   train_loss = 0.702  accuracy = 0.6875  auc = 0.5314009661835748\n",
      "2017-12-30T20:34:11.448244: Epoch   0 Batch 13000/28124   train_loss = 0.447  accuracy = 0.8125  auc = 0.796875\n",
      "2017-12-30T20:34:12.576583: Epoch   0 Batch 13025/28124   train_loss = 0.492  accuracy = 0.75  auc = 0.8227272727272728\n",
      "2017-12-30T20:34:13.686341: Epoch   0 Batch 13050/28124   train_loss = 0.510  accuracy = 0.75  auc = 0.78125\n",
      "2017-12-30T20:34:14.816883: Epoch   0 Batch 13075/28124   train_loss = 0.472  accuracy = 0.78125  auc = 0.7604166666666666\n",
      "2017-12-30T20:34:15.943971: Epoch   0 Batch 13100/28124   train_loss = 0.575  accuracy = 0.71875  auc = 0.6354166666666667\n",
      "2017-12-30T20:34:17.096853: Epoch   0 Batch 13125/28124   train_loss = 0.573  accuracy = 0.71875  auc = 0.7227272727272728\n",
      "2017-12-30T20:34:18.230191: Epoch   0 Batch 13150/28124   train_loss = 0.391  accuracy = 0.84375  auc = 0.8141025641025641\n",
      "2017-12-30T20:34:19.364634: Epoch   0 Batch 13175/28124   train_loss = 0.667  accuracy = 0.6875  auc = 0.8117647058823529\n",
      "2017-12-30T20:34:20.513062: Epoch   0 Batch 13200/28124   train_loss = 0.519  accuracy = 0.71875  auc = 0.8268398268398268\n",
      "2017-12-30T20:34:21.650878: Epoch   0 Batch 13225/28124   train_loss = 0.470  accuracy = 0.6875  auc = 0.8636363636363635\n",
      "2017-12-30T20:34:22.782034: Epoch   0 Batch 13250/28124   train_loss = 0.422  accuracy = 0.8125  auc = 0.8831168831168832\n",
      "2017-12-30T20:34:23.929266: Epoch   0 Batch 13275/28124   train_loss = 0.378  accuracy = 0.84375  auc = 0.8489583333333334\n",
      "2017-12-30T20:34:25.129023: Epoch   0 Batch 13300/28124   train_loss = 0.398  accuracy = 0.8125  auc = 0.9227272727272727\n",
      "2017-12-30T20:34:26.270671: Epoch   0 Batch 13325/28124   train_loss = 0.541  accuracy = 0.71875  auc = 0.5777777777777778\n",
      "2017-12-30T20:34:27.404116: Epoch   0 Batch 13350/28124   train_loss = 0.677  accuracy = 0.65625  auc = 0.5416666666666667\n",
      "2017-12-30T20:34:28.524764: Epoch   0 Batch 13375/28124   train_loss = 0.323  accuracy = 0.875  auc = 0.8592592592592594\n",
      "2017-12-30T20:34:29.670234: Epoch   0 Batch 13400/28124   train_loss = 0.470  accuracy = 0.75  auc = 0.8177083333333333\n",
      "2017-12-30T20:34:30.802267: Epoch   0 Batch 13425/28124   train_loss = 0.568  accuracy = 0.84375  auc = 0.41481481481481486\n",
      "2017-12-30T20:34:31.932461: Epoch   0 Batch 13450/28124   train_loss = 0.385  accuracy = 0.84375  auc = 0.8076923076923077\n",
      "2017-12-30T20:34:33.105136: Epoch   0 Batch 13475/28124   train_loss = 0.307  accuracy = 0.90625  auc = 0.8666666666666667\n",
      "2017-12-30T20:34:34.236578: Epoch   0 Batch 13500/28124   train_loss = 0.637  accuracy = 0.59375  auc = 0.6916666666666667\n",
      "2017-12-30T20:34:35.364469: Epoch   0 Batch 13525/28124   train_loss = 0.470  accuracy = 0.71875  auc = 0.8545454545454545\n",
      "2017-12-30T20:34:36.492924: Epoch   0 Batch 13550/28124   train_loss = 0.517  accuracy = 0.71875  auc = 0.8461538461538461\n",
      "2017-12-30T20:34:37.637065: Epoch   0 Batch 13575/28124   train_loss = 0.347  accuracy = 0.84375  auc = 0.90625\n",
      "2017-12-30T20:34:38.771128: Epoch   0 Batch 13600/28124   train_loss = 0.340  accuracy = 0.8125  auc = 0.8592592592592593\n",
      "2017-12-30T20:34:39.928472: Epoch   0 Batch 13625/28124   train_loss = 0.454  accuracy = 0.78125  auc = 0.7864583333333333\n",
      "2017-12-30T20:34:41.096631: Epoch   0 Batch 13650/28124   train_loss = 0.466  accuracy = 0.84375  auc = 0.7257142857142856\n",
      "2017-12-30T20:34:42.252997: Epoch   0 Batch 13675/28124   train_loss = 0.545  accuracy = 0.71875  auc = 0.75\n",
      "2017-12-30T20:34:43.408324: Epoch   0 Batch 13700/28124   train_loss = 0.647  accuracy = 0.71875  auc = 0.6454545454545455\n",
      "2017-12-30T20:34:44.551820: Epoch   0 Batch 13725/28124   train_loss = 0.360  accuracy = 0.84375  auc = 0.5833333333333334\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:34:45.696368: Epoch   0 Batch 13750/28124   train_loss = 0.299  accuracy = 0.90625  auc = 0.9017857142857143\n",
      "2017-12-30T20:34:46.829420: Epoch   0 Batch 13775/28124   train_loss = 0.306  accuracy = 0.90625  auc = 0.923076923076923\n",
      "2017-12-30T20:34:47.950245: Epoch   0 Batch 13800/28124   train_loss = 0.489  accuracy = 0.75  auc = 0.821256038647343\n",
      "2017-12-30T20:34:49.132772: Epoch   0 Batch 13825/28124   train_loss = 0.488  accuracy = 0.78125  auc = 0.7863636363636363\n",
      "2017-12-30T20:34:50.273158: Epoch   0 Batch 13850/28124   train_loss = 0.478  accuracy = 0.75  auc = 0.7428571428571429\n",
      "2017-12-30T20:34:51.471495: Epoch   0 Batch 13875/28124   train_loss = 0.346  accuracy = 0.84375  auc = 0.9227053140096618\n",
      "2017-12-30T20:34:52.594495: Epoch   0 Batch 13900/28124   train_loss = 0.395  accuracy = 0.84375  auc = 0.859375\n",
      "2017-12-30T20:34:53.733378: Epoch   0 Batch 13925/28124   train_loss = 0.452  accuracy = 0.8125  auc = 0.8363636363636363\n",
      "2017-12-30T20:34:54.850680: Epoch   0 Batch 13950/28124   train_loss = 0.347  accuracy = 0.84375  auc = 0.90625\n",
      "2017-12-30T20:34:56.004258: Epoch   0 Batch 13975/28124   train_loss = 0.545  accuracy = 0.71875  auc = 0.6057142857142856\n",
      "2017-12-30T20:34:57.179502: Epoch   0 Batch 14000/28124   train_loss = 0.370  accuracy = 0.84375  auc = 0.8342857142857143\n",
      "2017-12-30T20:34:58.308531: Epoch   0 Batch 14025/28124   train_loss = 0.544  accuracy = 0.78125  auc = 0.5512820512820513\n",
      "2017-12-30T20:34:59.442776: Epoch   0 Batch 14050/28124   train_loss = 0.421  accuracy = 0.8125  auc = 0.8072916666666667\n",
      "2017-12-30T20:35:00.586307: Epoch   0 Batch 14075/28124   train_loss = 0.466  accuracy = 0.78125  auc = 0.7085714285714285\n",
      "2017-12-30T20:35:01.711864: Epoch   0 Batch 14100/28124   train_loss = 0.478  accuracy = 0.75  auc = 0.8212560386473431\n",
      "2017-12-30T20:35:02.848272: Epoch   0 Batch 14125/28124   train_loss = 0.550  accuracy = 0.6875  auc = 0.6908212560386473\n",
      "2017-12-30T20:35:03.973114: Epoch   0 Batch 14150/28124   train_loss = 0.522  accuracy = 0.75  auc = 0.6927083333333333\n",
      "2017-12-30T20:35:05.140010: Epoch   0 Batch 14175/28124   train_loss = 0.418  accuracy = 0.84375  auc = 0.7884615384615384\n",
      "2017-12-30T20:35:06.269206: Epoch   0 Batch 14200/28124   train_loss = 0.467  accuracy = 0.75  auc = 0.8318181818181818\n",
      "2017-12-30T20:35:07.396511: Epoch   0 Batch 14225/28124   train_loss = 0.313  accuracy = 0.875  auc = 0.9635416666666667\n",
      "2017-12-30T20:35:08.527622: Epoch   0 Batch 14250/28124   train_loss = 0.429  accuracy = 0.8125  auc = 0.7828571428571429\n",
      "2017-12-30T20:35:09.656276: Epoch   0 Batch 14275/28124   train_loss = 0.468  accuracy = 0.78125  auc = 0.8225108225108225\n",
      "2017-12-30T20:35:10.783846: Epoch   0 Batch 14300/28124   train_loss = 0.349  accuracy = 0.875  auc = 0.942857142857143\n",
      "2017-12-30T20:35:11.920264: Epoch   0 Batch 14325/28124   train_loss = 0.429  accuracy = 0.8125  auc = 0.7692307692307692\n",
      "2017-12-30T20:35:13.071708: Epoch   0 Batch 14350/28124   train_loss = 0.536  accuracy = 0.8125  auc = 0.761904761904762\n",
      "2017-12-30T20:35:14.226533: Epoch   0 Batch 14375/28124   train_loss = 0.239  accuracy = 0.9375  auc = 0.9615384615384615\n",
      "2017-12-30T20:35:15.352652: Epoch   0 Batch 14400/28124   train_loss = 0.521  accuracy = 0.6875  auc = 0.8\n",
      "2017-12-30T20:35:16.483504: Epoch   0 Batch 14425/28124   train_loss = 0.490  accuracy = 0.75  auc = 0.8311688311688311\n",
      "2017-12-30T20:35:17.623887: Epoch   0 Batch 14450/28124   train_loss = 0.300  accuracy = 0.9375  auc = 0.9553571428571429\n",
      "2017-12-30T20:35:18.757388: Epoch   0 Batch 14475/28124   train_loss = 0.418  accuracy = 0.8125  auc = 0.8177083333333333\n",
      "2017-12-30T20:35:19.886326: Epoch   0 Batch 14500/28124   train_loss = 0.385  accuracy = 0.84375  auc = 0.875\n",
      "2017-12-30T20:35:21.033817: Epoch   0 Batch 14525/28124   train_loss = 0.548  accuracy = 0.6875  auc = 0.8375\n",
      "2017-12-30T20:35:22.193483: Epoch   0 Batch 14550/28124   train_loss = 0.402  accuracy = 0.84375  auc = 0.7555555555555555\n",
      "2017-12-30T20:35:23.311083: Epoch   0 Batch 14575/28124   train_loss = 0.437  accuracy = 0.8125  auc = 0.5517241379310345\n",
      "2017-12-30T20:35:24.431683: Epoch   0 Batch 14600/28124   train_loss = 0.446  accuracy = 0.71875  auc = 0.8545454545454545\n",
      "2017-12-30T20:35:25.563694: Epoch   0 Batch 14625/28124   train_loss = 0.454  accuracy = 0.84375  auc = 0.8227272727272728\n",
      "2017-12-30T20:35:26.713450: Epoch   0 Batch 14650/28124   train_loss = 0.372  accuracy = 0.78125  auc = 0.9818181818181818\n",
      "2017-12-30T20:35:27.836323: Epoch   0 Batch 14675/28124   train_loss = 0.410  accuracy = 0.875  auc = 0.7628205128205128\n",
      "2017-12-30T20:35:28.982296: Epoch   0 Batch 14700/28124   train_loss = 0.501  accuracy = 0.625  auc = 0.923076923076923\n",
      "2017-12-30T20:35:30.147620: Epoch   0 Batch 14725/28124   train_loss = 0.495  accuracy = 0.71875  auc = 0.7954545454545454\n",
      "2017-12-30T20:35:31.295035: Epoch   0 Batch 14750/28124   train_loss = 0.310  accuracy = 0.90625  auc = 0.8525641025641025\n",
      "2017-12-30T20:35:32.439857: Epoch   0 Batch 14775/28124   train_loss = 0.433  accuracy = 0.78125  auc = 0.8454106280193237\n",
      "2017-12-30T20:35:33.574933: Epoch   0 Batch 14800/28124   train_loss = 0.430  accuracy = 0.75  auc = 0.890625\n",
      "2017-12-30T20:35:35.143465: Epoch   0 Batch 14825/28124   train_loss = 0.392  accuracy = 0.875  auc = 0.8400000000000001\n",
      "2017-12-30T20:35:36.832573: Epoch   0 Batch 14850/28124   train_loss = 0.422  accuracy = 0.875  auc = 0.7771428571428571\n",
      "2017-12-30T20:35:38.665026: Epoch   0 Batch 14875/28124   train_loss = 0.596  accuracy = 0.75  auc = 0.6863636363636364\n",
      "2017-12-30T20:35:39.833701: Epoch   0 Batch 14900/28124   train_loss = 0.377  accuracy = 0.78125  auc = 0.961038961038961\n",
      "2017-12-30T20:35:41.486416: Epoch   0 Batch 14925/28124   train_loss = 0.358  accuracy = 0.875  auc = 0.84\n",
      "2017-12-30T20:35:42.849553: Epoch   0 Batch 14950/28124   train_loss = 0.452  accuracy = 0.6875  auc = 0.777142857142857\n",
      "2017-12-30T20:35:44.068619: Epoch   0 Batch 14975/28124   train_loss = 0.442  accuracy = 0.8125  auc = 0.796875\n",
      "2017-12-30T20:35:45.224271: Epoch   0 Batch 15000/28124   train_loss = 0.373  accuracy = 0.9375  auc = 0.6517857142857143\n",
      "2017-12-30T20:35:46.350984: Epoch   0 Batch 15025/28124   train_loss = 0.390  accuracy = 0.75  auc = 0.8541666666666666\n",
      "2017-12-30T20:35:47.483757: Epoch   0 Batch 15050/28124   train_loss = 0.309  accuracy = 0.8125  auc = 0.9714285714285715\n",
      "2017-12-30T20:35:48.632473: Epoch   0 Batch 15075/28124   train_loss = 0.702  accuracy = 0.59375  auc = 0.7056277056277056\n",
      "2017-12-30T20:35:49.790744: Epoch   0 Batch 15100/28124   train_loss = 0.527  accuracy = 0.75  auc = 0.6914285714285714\n",
      "2017-12-30T20:35:50.953039: Epoch   0 Batch 15125/28124   train_loss = 0.448  accuracy = 0.84375  auc = 0.76\n",
      "2017-12-30T20:35:52.122623: Epoch   0 Batch 15150/28124   train_loss = 0.487  accuracy = 0.8125  auc = 0.7542857142857142\n",
      "2017-12-30T20:35:53.274779: Epoch   0 Batch 15175/28124   train_loss = 0.699  accuracy = 0.65625  auc = 0.6\n",
      "2017-12-30T20:35:54.401976: Epoch   0 Batch 15200/28124   train_loss = 0.375  accuracy = 0.875  auc = 0.8599033816425121\n",
      "2017-12-30T20:35:55.541814: Epoch   0 Batch 15225/28124   train_loss = 0.386  accuracy = 0.84375  auc = 0.890909090909091\n",
      "2017-12-30T20:35:56.670597: Epoch   0 Batch 15250/28124   train_loss = 0.579  accuracy = 0.71875  auc = 0.7489177489177489\n",
      "2017-12-30T20:35:57.800403: Epoch   0 Batch 15275/28124   train_loss = 0.322  accuracy = 0.84375  auc = 0.8222222222222222\n",
      "2017-12-30T20:35:58.940171: Epoch   0 Batch 15300/28124   train_loss = 0.558  accuracy = 0.71875  auc = 0.8492063492063492\n",
      "2017-12-30T20:36:00.116945: Epoch   0 Batch 15325/28124   train_loss = 0.400  accuracy = 0.84375  auc = 0.8125\n",
      "2017-12-30T20:36:01.256887: Epoch   0 Batch 15350/28124   train_loss = 0.437  accuracy = 0.8125  auc = 0.78125\n",
      "2017-12-30T20:36:02.400327: Epoch   0 Batch 15375/28124   train_loss = 0.619  accuracy = 0.71875  auc = 0.5989583333333334\n",
      "2017-12-30T20:36:03.540028: Epoch   0 Batch 15400/28124   train_loss = 0.381  accuracy = 0.78125  auc = 0.9033816425120772\n",
      "2017-12-30T20:36:04.671747: Epoch   0 Batch 15425/28124   train_loss = 0.422  accuracy = 0.78125  auc = 0.8229166666666666\n",
      "2017-12-30T20:36:05.797793: Epoch   0 Batch 15450/28124   train_loss = 0.435  accuracy = 0.8125  auc = 0.7542857142857142\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:36:06.958965: Epoch   0 Batch 15475/28124   train_loss = 0.536  accuracy = 0.78125  auc = 0.7294685990338163\n",
      "2017-12-30T20:36:08.124778: Epoch   0 Batch 15500/28124   train_loss = 0.459  accuracy = 0.8125  auc = 0.7708333333333334\n",
      "2017-12-30T20:36:09.260707: Epoch   0 Batch 15525/28124   train_loss = 0.343  accuracy = 0.875  auc = 0.8782051282051282\n",
      "2017-12-30T20:36:10.406348: Epoch   0 Batch 15550/28124   train_loss = 0.393  accuracy = 0.84375  auc = 0.9090909090909091\n",
      "2017-12-30T20:36:11.533162: Epoch   0 Batch 15575/28124   train_loss = 0.403  accuracy = 0.8125  auc = 0.8228571428571428\n",
      "2017-12-30T20:36:12.677944: Epoch   0 Batch 15600/28124   train_loss = 0.591  accuracy = 0.6875  auc = 0.6183574879227053\n",
      "2017-12-30T20:36:13.847581: Epoch   0 Batch 15625/28124   train_loss = 0.411  accuracy = 0.90625  auc = 0.8695652173913044\n",
      "2017-12-30T20:36:15.000506: Epoch   0 Batch 15650/28124   train_loss = 0.351  accuracy = 0.8125  auc = 0.8074074074074074\n",
      "2017-12-30T20:36:16.186429: Epoch   0 Batch 15675/28124   train_loss = 0.323  accuracy = 0.90625  auc = 0.8906250000000001\n",
      "2017-12-30T20:36:17.321066: Epoch   0 Batch 15700/28124   train_loss = 0.327  accuracy = 0.84375  auc = 0.9428571428571428\n",
      "2017-12-30T20:36:18.458481: Epoch   0 Batch 15725/28124   train_loss = 0.403  accuracy = 0.8125  auc = 0.7037037037037038\n",
      "2017-12-30T20:36:19.599399: Epoch   0 Batch 15750/28124   train_loss = 0.367  accuracy = 0.8125  auc = 0.8525641025641025\n",
      "2017-12-30T20:36:20.744800: Epoch   0 Batch 15775/28124   train_loss = 0.360  accuracy = 0.84375  auc = 0.8653846153846154\n",
      "2017-12-30T20:36:21.899366: Epoch   0 Batch 15800/28124   train_loss = 0.340  accuracy = 0.90625  auc = 0.9565217391304348\n",
      "2017-12-30T20:36:23.081242: Epoch   0 Batch 15825/28124   train_loss = 0.677  accuracy = 0.6875  auc = 0.7125\n",
      "2017-12-30T20:36:24.234539: Epoch   0 Batch 15850/28124   train_loss = 0.434  accuracy = 0.78125  auc = 0.7692307692307692\n",
      "2017-12-30T20:36:25.374592: Epoch   0 Batch 15875/28124   train_loss = 0.299  accuracy = 0.90625  auc = 0.9407407407407409\n",
      "2017-12-30T20:36:26.522397: Epoch   0 Batch 15900/28124   train_loss = 0.699  accuracy = 0.6875  auc = 0.6363636363636364\n",
      "2017-12-30T20:36:27.654404: Epoch   0 Batch 15925/28124   train_loss = 0.416  accuracy = 0.8125  auc = 0.7756410256410257\n",
      "2017-12-30T20:36:28.788677: Epoch   0 Batch 15950/28124   train_loss = 0.334  accuracy = 0.875  auc = 0.8971428571428571\n",
      "2017-12-30T20:36:29.941915: Epoch   0 Batch 15975/28124   train_loss = 0.551  accuracy = 0.75  auc = 0.6956521739130435\n",
      "2017-12-30T20:36:31.175572: Epoch   0 Batch 16000/28124   train_loss = 0.469  accuracy = 0.78125  auc = 0.7291666666666667\n",
      "2017-12-30T20:36:32.299901: Epoch   0 Batch 16025/28124   train_loss = 0.259  accuracy = 0.875  auc = 0.9285714285714286\n",
      "2017-12-30T20:36:33.420293: Epoch   0 Batch 16050/28124   train_loss = 0.300  accuracy = 0.9375  auc = 0.8846153846153847\n",
      "2017-12-30T20:36:34.564686: Epoch   0 Batch 16075/28124   train_loss = 0.387  accuracy = 0.8125  auc = 0.8961038961038961\n",
      "2017-12-30T20:36:35.696611: Epoch   0 Batch 16100/28124   train_loss = 0.433  accuracy = 0.84375  auc = 0.7760416666666667\n",
      "2017-12-30T20:36:36.830546: Epoch   0 Batch 16125/28124   train_loss = 0.326  accuracy = 0.84375  auc = 0.8444444444444446\n",
      "2017-12-30T20:36:37.998148: Epoch   0 Batch 16150/28124   train_loss = 0.534  accuracy = 0.78125  auc = 0.7636363636363637\n",
      "2017-12-30T20:36:39.153945: Epoch   0 Batch 16175/28124   train_loss = 0.431  accuracy = 0.84375  auc = 0.6538461538461539\n",
      "2017-12-30T20:36:40.287373: Epoch   0 Batch 16200/28124   train_loss = 0.438  accuracy = 0.78125  auc = 0.7828571428571428\n",
      "2017-12-30T20:36:41.438931: Epoch   0 Batch 16225/28124   train_loss = 0.572  accuracy = 0.65625  auc = 0.8095238095238095\n",
      "2017-12-30T20:36:42.577918: Epoch   0 Batch 16250/28124   train_loss = 0.410  accuracy = 0.78125  auc = 0.6964285714285714\n",
      "2017-12-30T20:36:43.706414: Epoch   0 Batch 16275/28124   train_loss = 0.178  accuracy = 0.96875  auc = 0.9666666666666667\n",
      "2017-12-30T20:36:44.856416: Epoch   0 Batch 16300/28124   train_loss = 0.408  accuracy = 0.84375  auc = 0.7884615384615384\n",
      "2017-12-30T20:36:45.985192: Epoch   0 Batch 16325/28124   train_loss = 0.378  accuracy = 0.8125  auc = 0.862857142857143\n",
      "2017-12-30T20:36:47.138748: Epoch   0 Batch 16350/28124   train_loss = 0.392  accuracy = 0.8125  auc = 0.8342857142857143\n",
      "2017-12-30T20:36:48.267775: Epoch   0 Batch 16375/28124   train_loss = 0.242  accuracy = 0.96875  auc = 0.9166666666666667\n",
      "2017-12-30T20:36:49.379407: Epoch   0 Batch 16400/28124   train_loss = 0.456  accuracy = 0.78125  auc = 0.75\n",
      "2017-12-30T20:36:50.521399: Epoch   0 Batch 16425/28124   train_loss = 0.429  accuracy = 0.75  auc = 0.7692307692307692\n",
      "2017-12-30T20:36:51.696869: Epoch   0 Batch 16450/28124   train_loss = 0.495  accuracy = 0.71875  auc = 0.8225108225108225\n",
      "2017-12-30T20:36:52.848719: Epoch   0 Batch 16475/28124   train_loss = 0.326  accuracy = 0.90625  auc = 0.8846153846153846\n",
      "2017-12-30T20:36:53.979191: Epoch   0 Batch 16500/28124   train_loss = 0.335  accuracy = 0.875  auc = 0.8525641025641026\n",
      "2017-12-30T20:36:55.152096: Epoch   0 Batch 16525/28124   train_loss = 0.509  accuracy = 0.6875  auc = 0.8484848484848484\n",
      "2017-12-30T20:36:56.304431: Epoch   0 Batch 16550/28124   train_loss = 0.476  accuracy = 0.71875  auc = 0.8067632850241546\n",
      "2017-12-30T20:36:57.790381: Epoch   0 Batch 16575/28124   train_loss = 0.504  accuracy = 0.75  auc = 0.790909090909091\n",
      "2017-12-30T20:36:59.762652: Epoch   0 Batch 16600/28124   train_loss = 0.254  accuracy = 0.9375  auc = 0.9714285714285714\n",
      "2017-12-30T20:37:01.467466: Epoch   0 Batch 16625/28124   train_loss = 0.452  accuracy = 0.8125  auc = 0.7657142857142857\n",
      "2017-12-30T20:37:02.678385: Epoch   0 Batch 16650/28124   train_loss = 0.378  accuracy = 0.78125  auc = 0.8697916666666666\n",
      "2017-12-30T20:37:04.554916: Epoch   0 Batch 16675/28124   train_loss = 0.441  accuracy = 0.78125  auc = 0.7714285714285714\n",
      "2017-12-30T20:37:06.525630: Epoch   0 Batch 16700/28124   train_loss = 0.375  accuracy = 0.75  auc = 0.9033816425120772\n",
      "2017-12-30T20:37:08.066925: Epoch   0 Batch 16725/28124   train_loss = 0.209  accuracy = 1.0  auc = 1.0\n",
      "2017-12-30T20:37:09.971024: Epoch   0 Batch 16750/28124   train_loss = 0.277  accuracy = 0.9375  auc = 0.9423076923076923\n",
      "2017-12-30T20:37:11.571519: Epoch   0 Batch 16775/28124   train_loss = 0.428  accuracy = 0.78125  auc = 0.8489583333333333\n",
      "2017-12-30T20:37:12.940991: Epoch   0 Batch 16800/28124   train_loss = 0.335  accuracy = 0.84375  auc = 0.9294871794871794\n",
      "2017-12-30T20:37:14.116363: Epoch   0 Batch 16825/28124   train_loss = 0.336  accuracy = 0.9375  auc = 0.8125\n",
      "2017-12-30T20:37:15.259945: Epoch   0 Batch 16850/28124   train_loss = 0.510  accuracy = 0.71875  auc = 0.7878787878787878\n",
      "2017-12-30T20:37:16.401557: Epoch   0 Batch 16875/28124   train_loss = 0.377  accuracy = 0.875  auc = 0.9033816425120773\n",
      "2017-12-30T20:37:17.526298: Epoch   0 Batch 16900/28124   train_loss = 0.364  accuracy = 0.8125  auc = 0.9227053140096618\n",
      "2017-12-30T20:37:18.653796: Epoch   0 Batch 16925/28124   train_loss = 0.425  accuracy = 0.78125  auc = 0.8125\n",
      "2017-12-30T20:37:19.786550: Epoch   0 Batch 16950/28124   train_loss = 0.399  accuracy = 0.8125  auc = 0.875\n",
      "2017-12-30T20:37:20.937084: Epoch   0 Batch 16975/28124   train_loss = 0.459  accuracy = 0.78125  auc = 0.7826086956521738\n",
      "2017-12-30T20:37:22.100539: Epoch   0 Batch 17000/28124   train_loss = 0.424  accuracy = 0.78125  auc = 0.8454106280193237\n",
      "2017-12-30T20:37:23.225641: Epoch   0 Batch 17025/28124   train_loss = 0.627  accuracy = 0.75  auc = 0.757085020242915\n",
      "2017-12-30T20:37:24.351101: Epoch   0 Batch 17050/28124   train_loss = 0.370  accuracy = 0.875  auc = 0.9272727272727272\n",
      "2017-12-30T20:37:25.495102: Epoch   0 Batch 17075/28124   train_loss = 0.665  accuracy = 0.65625  auc = 0.6277056277056278\n",
      "2017-12-30T20:37:26.635962: Epoch   0 Batch 17100/28124   train_loss = 0.308  accuracy = 0.90625  auc = 0.9516908212560387\n",
      "2017-12-30T20:37:27.761201: Epoch   0 Batch 17125/28124   train_loss = 0.476  accuracy = 0.75  auc = 0.7604166666666666\n",
      "2017-12-30T20:37:28.917051: Epoch   0 Batch 17150/28124   train_loss = 0.415  accuracy = 0.875  auc = 0.8114285714285713\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:37:30.089875: Epoch   0 Batch 17175/28124   train_loss = 0.323  accuracy = 0.8125  auc = 0.9199999999999999\n",
      "2017-12-30T20:37:31.243488: Epoch   0 Batch 17200/28124   train_loss = 0.455  accuracy = 0.71875  auc = 0.76\n",
      "2017-12-30T20:37:32.370434: Epoch   0 Batch 17225/28124   train_loss = 0.417  accuracy = 0.875  auc = 0.6592592592592592\n",
      "2017-12-30T20:37:33.491540: Epoch   0 Batch 17250/28124   train_loss = 0.467  accuracy = 0.71875  auc = 0.8020833333333334\n",
      "2017-12-30T20:37:34.628755: Epoch   0 Batch 17275/28124   train_loss = 0.498  accuracy = 0.75  auc = 0.7447916666666666\n",
      "2017-12-30T20:37:35.779692: Epoch   0 Batch 17300/28124   train_loss = 0.600  accuracy = 0.6875  auc = 0.6545454545454545\n",
      "2017-12-30T20:37:36.936660: Epoch   0 Batch 17325/28124   train_loss = 0.485  accuracy = 0.8125  auc = 0.8363636363636363\n",
      "2017-12-30T20:37:38.136824: Epoch   0 Batch 17350/28124   train_loss = 0.294  accuracy = 0.84375  auc = 0.911111111111111\n",
      "2017-12-30T20:37:39.263421: Epoch   0 Batch 17375/28124   train_loss = 0.452  accuracy = 0.75  auc = 0.8502415458937198\n",
      "2017-12-30T20:37:40.405392: Epoch   0 Batch 17400/28124   train_loss = 0.383  accuracy = 0.84375  auc = 0.9045454545454545\n",
      "2017-12-30T20:37:41.545418: Epoch   0 Batch 17425/28124   train_loss = 0.430  accuracy = 0.78125  auc = 0.8514285714285714\n",
      "2017-12-30T20:37:42.670577: Epoch   0 Batch 17450/28124   train_loss = 0.434  accuracy = 0.84375  auc = 0.8260869565217391\n",
      "2017-12-30T20:37:43.776269: Epoch   0 Batch 17475/28124   train_loss = 0.423  accuracy = 0.78125  auc = 0.8772727272727273\n",
      "2017-12-30T20:37:44.918115: Epoch   0 Batch 17500/28124   train_loss = 0.364  accuracy = 0.8125  auc = 0.890625\n",
      "2017-12-30T20:37:46.093828: Epoch   0 Batch 17525/28124   train_loss = 0.422  accuracy = 0.84375  auc = 0.7628205128205128\n",
      "2017-12-30T20:37:47.219243: Epoch   0 Batch 17550/28124   train_loss = 0.393  accuracy = 0.84375  auc = 0.8385416666666667\n",
      "2017-12-30T20:37:48.350731: Epoch   0 Batch 17575/28124   train_loss = 0.388  accuracy = 0.78125  auc = 0.8685714285714285\n",
      "2017-12-30T20:37:49.468904: Epoch   0 Batch 17600/28124   train_loss = 0.485  accuracy = 0.71875  auc = 0.8541666666666667\n",
      "2017-12-30T20:37:50.595615: Epoch   0 Batch 17625/28124   train_loss = 0.432  accuracy = 0.75  auc = 0.8171428571428571\n",
      "2017-12-30T20:37:51.715993: Epoch   0 Batch 17650/28124   train_loss = 0.482  accuracy = 0.8125  auc = 0.6857142857142857\n",
      "2017-12-30T20:37:52.857596: Epoch   0 Batch 17675/28124   train_loss = 0.415  accuracy = 0.78125  auc = 0.9083333333333333\n",
      "2017-12-30T20:37:53.989965: Epoch   0 Batch 17700/28124   train_loss = 0.301  accuracy = 0.90625  auc = 0.953125\n",
      "2017-12-30T20:37:55.176548: Epoch   0 Batch 17725/28124   train_loss = 0.457  accuracy = 0.78125  auc = 0.7239583333333334\n",
      "2017-12-30T20:37:56.322549: Epoch   0 Batch 17750/28124   train_loss = 0.398  accuracy = 0.90625  auc = 0.8550724637681159\n",
      "2017-12-30T20:37:57.452393: Epoch   0 Batch 17775/28124   train_loss = 0.419  accuracy = 0.875  auc = 0.7708333333333333\n",
      "2017-12-30T20:37:58.586772: Epoch   0 Batch 17800/28124   train_loss = 0.563  accuracy = 0.71875  auc = 0.8259109311740891\n",
      "2017-12-30T20:37:59.712014: Epoch   0 Batch 17825/28124   train_loss = 0.402  accuracy = 0.75  auc = 0.8857142857142857\n",
      "2017-12-30T20:38:00.842328: Epoch   0 Batch 17850/28124   train_loss = 0.238  accuracy = 0.9375  auc = 0.9310344827586207\n",
      "2017-12-30T20:38:01.968691: Epoch   0 Batch 17875/28124   train_loss = 0.495  accuracy = 0.78125  auc = 0.8354978354978355\n",
      "2017-12-30T20:38:03.128362: Epoch   0 Batch 17900/28124   train_loss = 0.361  accuracy = 0.8125  auc = 0.8974358974358974\n",
      "2017-12-30T20:38:04.256845: Epoch   0 Batch 17925/28124   train_loss = 0.374  accuracy = 0.90625  auc = 0.8854166666666667\n",
      "2017-12-30T20:38:05.360797: Epoch   0 Batch 17950/28124   train_loss = 0.470  accuracy = 0.78125  auc = 0.8484848484848485\n",
      "2017-12-30T20:38:06.490430: Epoch   0 Batch 17975/28124   train_loss = 0.498  accuracy = 0.71875  auc = 0.72\n",
      "2017-12-30T20:38:07.620653: Epoch   0 Batch 18000/28124   train_loss = 0.513  accuracy = 0.78125  auc = 0.8008658008658008\n",
      "2017-12-30T20:38:08.743861: Epoch   0 Batch 18025/28124   train_loss = 0.335  accuracy = 0.875  auc = 0.8740740740740741\n",
      "2017-12-30T20:38:09.883928: Epoch   0 Batch 18050/28124   train_loss = 0.583  accuracy = 0.8125  auc = 0.681159420289855\n",
      "2017-12-30T20:38:11.007106: Epoch   0 Batch 18075/28124   train_loss = 0.295  accuracy = 0.875  auc = 0.8666666666666667\n",
      "2017-12-30T20:38:12.157628: Epoch   0 Batch 18100/28124   train_loss = 0.402  accuracy = 0.84375  auc = 0.78125\n",
      "2017-12-30T20:38:13.291727: Epoch   0 Batch 18125/28124   train_loss = 0.384  accuracy = 0.78125  auc = 0.9437229437229437\n",
      "2017-12-30T20:38:14.417478: Epoch   0 Batch 18150/28124   train_loss = 0.417  accuracy = 0.84375  auc = 0.84375\n",
      "2017-12-30T20:38:15.561823: Epoch   0 Batch 18175/28124   train_loss = 0.345  accuracy = 0.84375  auc = 0.8525641025641025\n",
      "2017-12-30T20:38:16.687195: Epoch   0 Batch 18200/28124   train_loss = 0.462  accuracy = 0.78125  auc = 0.8268398268398268\n",
      "2017-12-30T20:38:17.821747: Epoch   0 Batch 18225/28124   train_loss = 0.644  accuracy = 0.6875  auc = 0.7045454545454546\n",
      "2017-12-30T20:38:18.966602: Epoch   0 Batch 18250/28124   train_loss = 0.477  accuracy = 0.8125  auc = 0.7584541062801933\n",
      "2017-12-30T20:38:20.141562: Epoch   0 Batch 18275/28124   train_loss = 0.479  accuracy = 0.78125  auc = 0.7135416666666666\n",
      "2017-12-30T20:38:21.308576: Epoch   0 Batch 18300/28124   train_loss = 0.523  accuracy = 0.71875  auc = 0.7835497835497836\n",
      "2017-12-30T20:38:22.453914: Epoch   0 Batch 18325/28124   train_loss = 0.381  accuracy = 0.84375  auc = 0.84\n",
      "2017-12-30T20:38:23.757845: Epoch   0 Batch 18350/28124   train_loss = 0.482  accuracy = 0.78125  auc = 0.7777777777777778\n",
      "2017-12-30T20:38:25.617958: Epoch   0 Batch 18375/28124   train_loss = 0.450  accuracy = 0.84375  auc = 0.8227272727272726\n",
      "2017-12-30T20:38:27.537926: Epoch   0 Batch 18400/28124   train_loss = 0.585  accuracy = 0.75  auc = 0.6763285024154589\n",
      "2017-12-30T20:38:28.738239: Epoch   0 Batch 18425/28124   train_loss = 0.328  accuracy = 0.875  auc = 0.96875\n",
      "2017-12-30T20:38:30.420288: Epoch   0 Batch 18450/28124   train_loss = 0.470  accuracy = 0.75  auc = 0.8357487922705313\n",
      "2017-12-30T20:38:31.649435: Epoch   0 Batch 18475/28124   train_loss = 0.344  accuracy = 0.875  auc = 0.8717948717948718\n",
      "2017-12-30T20:38:32.808453: Epoch   0 Batch 18500/28124   train_loss = 0.459  accuracy = 0.8125  auc = 0.8727272727272727\n",
      "2017-12-30T20:38:33.962031: Epoch   0 Batch 18525/28124   train_loss = 0.291  accuracy = 0.90625  auc = 0.9553571428571429\n",
      "2017-12-30T20:38:35.171146: Epoch   0 Batch 18550/28124   train_loss = 0.317  accuracy = 0.875  auc = 0.874074074074074\n",
      "2017-12-30T20:38:36.316422: Epoch   0 Batch 18575/28124   train_loss = 0.308  accuracy = 0.875  auc = 0.8296296296296296\n",
      "2017-12-30T20:38:37.477751: Epoch   0 Batch 18600/28124   train_loss = 0.415  accuracy = 0.78125  auc = 0.8697916666666667\n",
      "2017-12-30T20:38:38.666848: Epoch   0 Batch 18625/28124   train_loss = 0.613  accuracy = 0.75  auc = 0.7408906882591093\n",
      "2017-12-30T20:38:39.817495: Epoch   0 Batch 18650/28124   train_loss = 0.489  accuracy = 0.6875  auc = 0.72\n",
      "2017-12-30T20:38:40.955913: Epoch   0 Batch 18675/28124   train_loss = 0.482  accuracy = 0.71875  auc = 0.7922705314009661\n",
      "2017-12-30T20:38:42.146722: Epoch   0 Batch 18700/28124   train_loss = 0.515  accuracy = 0.75  auc = 0.7584541062801933\n",
      "2017-12-30T20:38:43.284391: Epoch   0 Batch 18725/28124   train_loss = 0.376  accuracy = 0.84375  auc = 0.75\n",
      "2017-12-30T20:38:44.433057: Epoch   0 Batch 18750/28124   train_loss = 0.400  accuracy = 0.8125  auc = 0.922077922077922\n",
      "2017-12-30T20:38:45.570317: Epoch   0 Batch 18775/28124   train_loss = 0.317  accuracy = 0.8125  auc = 0.8740740740740741\n",
      "2017-12-30T20:38:46.695696: Epoch   0 Batch 18800/28124   train_loss = 0.451  accuracy = 0.84375  auc = 0.48214285714285715\n",
      "2017-12-30T20:38:47.825892: Epoch   0 Batch 18825/28124   train_loss = 0.406  accuracy = 0.84375  auc = 0.84375\n",
      "2017-12-30T20:38:48.968529: Epoch   0 Batch 18850/28124   train_loss = 0.354  accuracy = 0.875  auc = 0.6666666666666666\n",
      "2017-12-30T20:38:50.155881: Epoch   0 Batch 18875/28124   train_loss = 0.363  accuracy = 0.8125  auc = 0.8857142857142857\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:38:51.341006: Epoch   0 Batch 18900/28124   train_loss = 0.468  accuracy = 0.78125  auc = 0.7428571428571428\n",
      "2017-12-30T20:38:52.495404: Epoch   0 Batch 18925/28124   train_loss = 0.600  accuracy = 0.71875  auc = 0.7181818181818181\n",
      "2017-12-30T20:38:53.647750: Epoch   0 Batch 18950/28124   train_loss = 0.583  accuracy = 0.65625  auc = 0.7732793522267207\n",
      "2017-12-30T20:38:54.780033: Epoch   0 Batch 18975/28124   train_loss = 0.462  accuracy = 0.71875  auc = 0.8987854251012145\n",
      "2017-12-30T20:38:55.952072: Epoch   0 Batch 19000/28124   train_loss = 0.591  accuracy = 0.6875  auc = 0.7705627705627704\n",
      "2017-12-30T20:38:57.135272: Epoch   0 Batch 19025/28124   train_loss = 0.457  accuracy = 0.8125  auc = 0.7828571428571428\n",
      "2017-12-30T20:38:58.280577: Epoch   0 Batch 19050/28124   train_loss = 0.443  accuracy = 0.8125  auc = 0.8787878787878789\n",
      "2017-12-30T20:38:59.422386: Epoch   0 Batch 19075/28124   train_loss = 0.421  accuracy = 0.8125  auc = 0.9090909090909091\n",
      "2017-12-30T20:39:00.577633: Epoch   0 Batch 19100/28124   train_loss = 0.320  accuracy = 0.84375  auc = 0.8814814814814815\n",
      "2017-12-30T20:39:01.700343: Epoch   0 Batch 19125/28124   train_loss = 0.472  accuracy = 0.8125  auc = 0.734375\n",
      "2017-12-30T20:39:02.834022: Epoch   0 Batch 19150/28124   train_loss = 0.582  accuracy = 0.65625  auc = 0.5448717948717949\n",
      "2017-12-30T20:39:03.965301: Epoch   0 Batch 19175/28124   train_loss = 0.613  accuracy = 0.75  auc = 0.5885416666666667\n",
      "2017-12-30T20:39:05.150053: Epoch   0 Batch 19200/28124   train_loss = 0.516  accuracy = 0.78125  auc = 0.7632850241545894\n",
      "2017-12-30T20:39:06.274304: Epoch   0 Batch 19225/28124   train_loss = 0.517  accuracy = 0.75  auc = 0.5256410256410255\n",
      "2017-12-30T20:39:07.421415: Epoch   0 Batch 19250/28124   train_loss = 0.431  accuracy = 0.78125  auc = 0.828125\n",
      "2017-12-30T20:39:08.585734: Epoch   0 Batch 19275/28124   train_loss = 0.439  accuracy = 0.84375  auc = 0.8177083333333333\n",
      "2017-12-30T20:39:09.703027: Epoch   0 Batch 19300/28124   train_loss = 0.295  accuracy = 0.875  auc = 0.9487179487179487\n",
      "2017-12-30T20:39:10.841237: Epoch   0 Batch 19325/28124   train_loss = 0.647  accuracy = 0.71875  auc = 0.6233766233766234\n",
      "2017-12-30T20:39:11.990575: Epoch   0 Batch 19350/28124   train_loss = 0.408  accuracy = 0.78125  auc = 0.7884615384615384\n",
      "2017-12-30T20:39:13.162986: Epoch   0 Batch 19375/28124   train_loss = 0.389  accuracy = 0.8125  auc = 0.9082125603864735\n",
      "2017-12-30T20:39:14.315412: Epoch   0 Batch 19400/28124   train_loss = 0.249  accuracy = 0.96875  auc = 0.9925925925925927\n",
      "2017-12-30T20:39:15.439405: Epoch   0 Batch 19425/28124   train_loss = 0.468  accuracy = 0.84375  auc = 0.6592592592592593\n",
      "2017-12-30T20:39:16.582369: Epoch   0 Batch 19450/28124   train_loss = 0.404  accuracy = 0.90625  auc = 0.6666666666666667\n",
      "2017-12-30T20:39:17.713299: Epoch   0 Batch 19475/28124   train_loss = 0.422  accuracy = 0.84375  auc = 0.7925925925925927\n",
      "2017-12-30T20:39:18.873467: Epoch   0 Batch 19500/28124   train_loss = 0.428  accuracy = 0.8125  auc = 0.8177083333333333\n",
      "2017-12-30T20:39:20.020657: Epoch   0 Batch 19525/28124   train_loss = 0.489  accuracy = 0.8125  auc = 0.7971014492753623\n",
      "2017-12-30T20:39:21.202846: Epoch   0 Batch 19550/28124   train_loss = 0.600  accuracy = 0.71875  auc = 0.7439613526570049\n",
      "2017-12-30T20:39:22.345234: Epoch   0 Batch 19575/28124   train_loss = 0.428  accuracy = 0.75  auc = 0.7771428571428571\n",
      "2017-12-30T20:39:23.490117: Epoch   0 Batch 19600/28124   train_loss = 0.474  accuracy = 0.8125  auc = 0.7777777777777778\n",
      "2017-12-30T20:39:24.616174: Epoch   0 Batch 19625/28124   train_loss = 0.512  accuracy = 0.75  auc = 0.7552083333333333\n",
      "2017-12-30T20:39:25.765345: Epoch   0 Batch 19650/28124   train_loss = 0.543  accuracy = 0.8125  auc = 0.6742857142857143\n",
      "2017-12-30T20:39:26.900455: Epoch   0 Batch 19675/28124   train_loss = 0.399  accuracy = 0.78125  auc = 0.8792270531400966\n",
      "2017-12-30T20:39:28.085375: Epoch   0 Batch 19700/28124   train_loss = 0.471  accuracy = 0.71875  auc = 0.7971014492753623\n",
      "2017-12-30T20:39:29.233420: Epoch   0 Batch 19725/28124   train_loss = 0.495  accuracy = 0.75  auc = 0.7552083333333334\n",
      "2017-12-30T20:39:30.365176: Epoch   0 Batch 19750/28124   train_loss = 0.444  accuracy = 0.8125  auc = 0.8019323671497585\n",
      "2017-12-30T20:39:31.511234: Epoch   0 Batch 19775/28124   train_loss = 0.423  accuracy = 0.78125  auc = 0.8541666666666667\n",
      "2017-12-30T20:39:32.655551: Epoch   0 Batch 19800/28124   train_loss = 0.464  accuracy = 0.6875  auc = 0.796875\n",
      "2017-12-30T20:39:33.806762: Epoch   0 Batch 19825/28124   train_loss = 0.678  accuracy = 0.71875  auc = 0.6045454545454545\n",
      "2017-12-30T20:39:34.936965: Epoch   0 Batch 19850/28124   train_loss = 0.497  accuracy = 0.75  auc = 0.8333333333333334\n",
      "2017-12-30T20:39:36.135220: Epoch   0 Batch 19875/28124   train_loss = 0.392  accuracy = 0.84375  auc = 0.8333333333333333\n",
      "2017-12-30T20:39:37.278396: Epoch   0 Batch 19900/28124   train_loss = 0.502  accuracy = 0.75  auc = 0.6153846153846154\n",
      "2017-12-30T20:39:38.471783: Epoch   0 Batch 19925/28124   train_loss = 0.533  accuracy = 0.6875  auc = 0.75\n",
      "2017-12-30T20:39:39.610095: Epoch   0 Batch 19950/28124   train_loss = 0.340  accuracy = 0.875  auc = 0.9033816425120773\n",
      "2017-12-30T20:39:40.746674: Epoch   0 Batch 19975/28124   train_loss = 0.425  accuracy = 0.8125  auc = 0.8599033816425121\n",
      "2017-12-30T20:39:41.897726: Epoch   0 Batch 20000/28124   train_loss = 0.334  accuracy = 0.84375  auc = 0.9427083333333333\n",
      "2017-12-30T20:39:43.094173: Epoch   0 Batch 20025/28124   train_loss = 0.475  accuracy = 0.75  auc = 0.8502415458937198\n",
      "2017-12-30T20:39:44.239743: Epoch   0 Batch 20050/28124   train_loss = 0.521  accuracy = 0.6875  auc = 0.8259109311740891\n",
      "2017-12-30T20:39:45.410410: Epoch   0 Batch 20075/28124   train_loss = 0.534  accuracy = 0.75  auc = 0.8340080971659919\n",
      "2017-12-30T20:39:46.550897: Epoch   0 Batch 20100/28124   train_loss = 0.320  accuracy = 0.84375  auc = 0.9423076923076923\n",
      "2017-12-30T20:39:47.677525: Epoch   0 Batch 20125/28124   train_loss = 0.437  accuracy = 0.84375  auc = 0.7243589743589743\n",
      "2017-12-30T20:39:48.810922: Epoch   0 Batch 20150/28124   train_loss = 0.627  accuracy = 0.71875  auc = 0.5893719806763285\n",
      "2017-12-30T20:39:49.956655: Epoch   0 Batch 20175/28124   train_loss = 0.299  accuracy = 0.84375  auc = 0.9322916666666666\n",
      "2017-12-30T20:39:51.148281: Epoch   0 Batch 20200/28124   train_loss = 0.403  accuracy = 0.8125  auc = 0.9272727272727272\n",
      "2017-12-30T20:39:52.283203: Epoch   0 Batch 20225/28124   train_loss = 0.347  accuracy = 0.875  auc = 0.9479166666666667\n",
      "2017-12-30T20:39:53.407127: Epoch   0 Batch 20250/28124   train_loss = 0.345  accuracy = 0.84375  auc = 0.9333333333333332\n",
      "2017-12-30T20:39:54.553114: Epoch   0 Batch 20275/28124   train_loss = 0.522  accuracy = 0.75  auc = 0.7681818181818182\n",
      "2017-12-30T20:39:55.675869: Epoch   0 Batch 20300/28124   train_loss = 0.447  accuracy = 0.8125  auc = 0.7037037037037037\n",
      "2017-12-30T20:39:56.833497: Epoch   0 Batch 20325/28124   train_loss = 0.498  accuracy = 0.71875  auc = 0.765625\n",
      "2017-12-30T20:39:57.959164: Epoch   0 Batch 20350/28124   train_loss = 0.676  accuracy = 0.65625  auc = 0.7333333333333334\n",
      "2017-12-30T20:39:59.143021: Epoch   0 Batch 20375/28124   train_loss = 0.424  accuracy = 0.875  auc = 0.8385416666666666\n",
      "2017-12-30T20:40:00.283518: Epoch   0 Batch 20400/28124   train_loss = 0.657  accuracy = 0.71875  auc = 0.5974025974025974\n",
      "2017-12-30T20:40:01.442020: Epoch   0 Batch 20425/28124   train_loss = 0.442  accuracy = 0.78125  auc = 0.8177083333333335\n",
      "2017-12-30T20:40:02.595741: Epoch   0 Batch 20450/28124   train_loss = 0.347  accuracy = 0.875  auc = 0.8846153846153846\n",
      "2017-12-30T20:40:03.735991: Epoch   0 Batch 20475/28124   train_loss = 0.392  accuracy = 0.78125  auc = 0.8985507246376812\n",
      "2017-12-30T20:40:04.881468: Epoch   0 Batch 20500/28124   train_loss = 0.493  accuracy = 0.875  auc = 0.5977011494252873\n",
      "2017-12-30T20:40:06.032360: Epoch   0 Batch 20525/28124   train_loss = 0.290  accuracy = 0.90625  auc = 0.875\n",
      "2017-12-30T20:40:07.222777: Epoch   0 Batch 20550/28124   train_loss = 0.458  accuracy = 0.78125  auc = 0.859090909090909\n",
      "2017-12-30T20:40:08.366049: Epoch   0 Batch 20575/28124   train_loss = 0.497  accuracy = 0.71875  auc = 0.8571428571428572\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:40:09.505327: Epoch   0 Batch 20600/28124   train_loss = 0.500  accuracy = 0.8125  auc = 0.8484848484848484\n",
      "2017-12-30T20:40:10.668858: Epoch   0 Batch 20625/28124   train_loss = 0.357  accuracy = 0.84375  auc = 0.8854166666666667\n",
      "2017-12-30T20:40:11.911985: Epoch   0 Batch 20650/28124   train_loss = 0.448  accuracy = 0.8125  auc = 0.7916666666666665\n",
      "2017-12-30T20:40:13.198948: Epoch   0 Batch 20675/28124   train_loss = 0.442  accuracy = 0.75  auc = 0.8590909090909091\n",
      "2017-12-30T20:40:14.349524: Epoch   0 Batch 20700/28124   train_loss = 0.506  accuracy = 0.75  auc = 0.75\n",
      "2017-12-30T20:40:15.511131: Epoch   0 Batch 20725/28124   train_loss = 0.440  accuracy = 0.78125  auc = 0.8363636363636364\n",
      "2017-12-30T20:40:16.649376: Epoch   0 Batch 20750/28124   train_loss = 0.461  accuracy = 0.71875  auc = 0.6444444444444445\n",
      "2017-12-30T20:40:17.787560: Epoch   0 Batch 20775/28124   train_loss = 0.466  accuracy = 0.78125  auc = 0.7760416666666667\n",
      "2017-12-30T20:40:18.938998: Epoch   0 Batch 20800/28124   train_loss = 0.473  accuracy = 0.75  auc = 0.8136363636363636\n",
      "2017-12-30T20:40:20.152975: Epoch   0 Batch 20825/28124   train_loss = 0.351  accuracy = 0.8125  auc = 0.9085714285714286\n",
      "2017-12-30T20:40:21.390418: Epoch   0 Batch 20850/28124   train_loss = 0.458  accuracy = 0.78125  auc = 0.8405797101449275\n",
      "2017-12-30T20:40:22.543144: Epoch   0 Batch 20875/28124   train_loss = 0.662  accuracy = 0.65625  auc = 0.6416666666666666\n",
      "2017-12-30T20:40:23.711078: Epoch   0 Batch 20900/28124   train_loss = 0.348  accuracy = 0.78125  auc = 0.8214285714285715\n",
      "2017-12-30T20:40:24.851888: Epoch   0 Batch 20925/28124   train_loss = 0.351  accuracy = 0.875  auc = 0.8914285714285715\n",
      "2017-12-30T20:40:25.997661: Epoch   0 Batch 20950/28124   train_loss = 0.445  accuracy = 0.78125  auc = 0.8787878787878788\n",
      "2017-12-30T20:40:27.184185: Epoch   0 Batch 20975/28124   train_loss = 0.428  accuracy = 0.84375  auc = 0.6296296296296297\n",
      "2017-12-30T20:40:28.319101: Epoch   0 Batch 21000/28124   train_loss = 0.448  accuracy = 0.8125  auc = 0.7760416666666667\n",
      "2017-12-30T20:40:29.472942: Epoch   0 Batch 21025/28124   train_loss = 0.316  accuracy = 0.90625  auc = 0.9166666666666666\n",
      "2017-12-30T20:40:30.622166: Epoch   0 Batch 21050/28124   train_loss = 0.432  accuracy = 0.8125  auc = 0.7942857142857143\n",
      "2017-12-30T20:40:31.771624: Epoch   0 Batch 21075/28124   train_loss = 0.507  accuracy = 0.71875  auc = 0.7584541062801933\n",
      "2017-12-30T20:40:32.933166: Epoch   0 Batch 21100/28124   train_loss = 0.410  accuracy = 0.78125  auc = 0.8171428571428572\n",
      "2017-12-30T20:40:34.106087: Epoch   0 Batch 21125/28124   train_loss = 0.411  accuracy = 0.875  auc = 0.75\n",
      "2017-12-30T20:40:35.266591: Epoch   0 Batch 21150/28124   train_loss = 0.415  accuracy = 0.8125  auc = 0.84\n",
      "2017-12-30T20:40:36.400481: Epoch   0 Batch 21175/28124   train_loss = 0.391  accuracy = 0.84375  auc = 0.8489583333333333\n",
      "2017-12-30T20:40:37.532103: Epoch   0 Batch 21200/28124   train_loss = 0.496  accuracy = 0.8125  auc = 0.5925925925925926\n",
      "2017-12-30T20:40:38.708544: Epoch   0 Batch 21225/28124   train_loss = 0.322  accuracy = 0.875  auc = 0.8850574712643677\n",
      "2017-12-30T20:40:39.840077: Epoch   0 Batch 21250/28124   train_loss = 0.451  accuracy = 0.84375  auc = 0.7948717948717948\n",
      "2017-12-30T20:40:40.987633: Epoch   0 Batch 21275/28124   train_loss = 0.540  accuracy = 0.78125  auc = 0.7101449275362319\n",
      "2017-12-30T20:40:42.161536: Epoch   0 Batch 21300/28124   train_loss = 0.435  accuracy = 0.71875  auc = 0.9541666666666667\n",
      "2017-12-30T20:40:43.299559: Epoch   0 Batch 21325/28124   train_loss = 0.415  accuracy = 0.8125  auc = 0.7589285714285714\n",
      "2017-12-30T20:40:44.439262: Epoch   0 Batch 21350/28124   train_loss = 0.313  accuracy = 0.84375  auc = 0.9657142857142856\n",
      "2017-12-30T20:40:45.572067: Epoch   0 Batch 21375/28124   train_loss = 0.442  accuracy = 0.75  auc = 0.8658008658008658\n",
      "2017-12-30T20:40:46.708647: Epoch   0 Batch 21400/28124   train_loss = 0.453  accuracy = 0.84375  auc = 0.8272727272727273\n",
      "2017-12-30T20:40:47.844271: Epoch   0 Batch 21425/28124   train_loss = 0.376  accuracy = 0.875  auc = 0.8444444444444444\n",
      "2017-12-30T20:40:48.984158: Epoch   0 Batch 21450/28124   train_loss = 0.432  accuracy = 0.84375  auc = 0.8658008658008658\n",
      "2017-12-30T20:40:50.157986: Epoch   0 Batch 21475/28124   train_loss = 0.597  accuracy = 0.75  auc = 0.7291666666666666\n",
      "2017-12-30T20:40:51.349878: Epoch   0 Batch 21500/28124   train_loss = 0.362  accuracy = 0.8125  auc = 0.9363636363636363\n",
      "2017-12-30T20:40:52.484863: Epoch   0 Batch 21525/28124   train_loss = 0.372  accuracy = 0.84375  auc = 0.7948717948717948\n",
      "2017-12-30T20:40:53.623423: Epoch   0 Batch 21550/28124   train_loss = 0.345  accuracy = 0.875  auc = 0.8628571428571428\n",
      "2017-12-30T20:40:54.746115: Epoch   0 Batch 21575/28124   train_loss = 0.357  accuracy = 0.875  auc = 0.8857142857142857\n",
      "2017-12-30T20:40:55.885293: Epoch   0 Batch 21600/28124   train_loss = 0.336  accuracy = 0.875  auc = 0.7666666666666666\n",
      "2017-12-30T20:40:57.055137: Epoch   0 Batch 21625/28124   train_loss = 0.451  accuracy = 0.84375  auc = 0.7555555555555555\n",
      "2017-12-30T20:40:58.179321: Epoch   0 Batch 21650/28124   train_loss = 0.493  accuracy = 0.71875  auc = 0.7777777777777778\n",
      "2017-12-30T20:40:59.318301: Epoch   0 Batch 21675/28124   train_loss = 0.346  accuracy = 0.84375  auc = 0.858974358974359\n",
      "2017-12-30T20:41:00.450688: Epoch   0 Batch 21700/28124   train_loss = 0.359  accuracy = 0.875  auc = 0.6875\n",
      "2017-12-30T20:41:01.571946: Epoch   0 Batch 21725/28124   train_loss = 0.376  accuracy = 0.78125  auc = 0.8397435897435898\n",
      "2017-12-30T20:41:02.711778: Epoch   0 Batch 21750/28124   train_loss = 0.531  accuracy = 0.71875  auc = 0.7004830917874396\n",
      "2017-12-30T20:41:03.923496: Epoch   0 Batch 21775/28124   train_loss = 0.472  accuracy = 0.8125  auc = 0.7395833333333334\n",
      "2017-12-30T20:41:05.095234: Epoch   0 Batch 21800/28124   train_loss = 0.426  accuracy = 0.875  auc = 0.7916666666666666\n",
      "2017-12-30T20:41:06.231716: Epoch   0 Batch 21825/28124   train_loss = 0.284  accuracy = 0.9375  auc = 0.9257142857142857\n",
      "2017-12-30T20:41:07.358665: Epoch   0 Batch 21850/28124   train_loss = 0.492  accuracy = 0.75  auc = 0.7818181818181817\n",
      "2017-12-30T20:41:08.484566: Epoch   0 Batch 21875/28124   train_loss = 0.516  accuracy = 0.75  auc = 0.7391304347826086\n",
      "2017-12-30T20:41:09.617951: Epoch   0 Batch 21900/28124   train_loss = 0.563  accuracy = 0.8125  auc = 0.6908212560386473\n",
      "2017-12-30T20:41:10.752328: Epoch   0 Batch 21925/28124   train_loss = 0.588  accuracy = 0.6875  auc = 0.7045454545454546\n",
      "2017-12-30T20:41:11.891949: Epoch   0 Batch 21950/28124   train_loss = 0.430  accuracy = 0.8125  auc = 0.7542857142857142\n",
      "2017-12-30T20:41:13.045139: Epoch   0 Batch 21975/28124   train_loss = 0.417  accuracy = 0.78125  auc = 0.8818181818181818\n",
      "2017-12-30T20:41:14.224898: Epoch   0 Batch 22000/28124   train_loss = 0.440  accuracy = 0.78125  auc = 0.8136363636363637\n",
      "2017-12-30T20:41:15.365304: Epoch   0 Batch 22025/28124   train_loss = 0.656  accuracy = 0.71875  auc = 0.7\n",
      "2017-12-30T20:41:16.511909: Epoch   0 Batch 22050/28124   train_loss = 0.667  accuracy = 0.6875  auc = 0.6060606060606061\n",
      "2017-12-30T20:41:17.632144: Epoch   0 Batch 22075/28124   train_loss = 0.519  accuracy = 0.71875  auc = 0.7975708502024292\n",
      "2017-12-30T20:41:18.756800: Epoch   0 Batch 22100/28124   train_loss = 0.426  accuracy = 0.75  auc = 0.8171428571428572\n",
      "2017-12-30T20:41:19.886960: Epoch   0 Batch 22125/28124   train_loss = 0.571  accuracy = 0.75  auc = 0.7975708502024291\n",
      "2017-12-30T20:41:21.044047: Epoch   0 Batch 22150/28124   train_loss = 0.410  accuracy = 0.8125  auc = 0.875\n",
      "2017-12-30T20:41:22.220128: Epoch   0 Batch 22175/28124   train_loss = 0.325  accuracy = 0.90625  auc = 0.9322916666666667\n",
      "2017-12-30T20:41:23.345307: Epoch   0 Batch 22200/28124   train_loss = 0.506  accuracy = 0.71875  auc = 0.8311688311688311\n",
      "2017-12-30T20:41:24.471799: Epoch   0 Batch 22225/28124   train_loss = 0.589  accuracy = 0.65625  auc = 0.6328502415458938\n",
      "2017-12-30T20:41:25.620184: Epoch   0 Batch 22250/28124   train_loss = 0.619  accuracy = 0.6875  auc = 0.6926406926406926\n",
      "2017-12-30T20:41:26.775445: Epoch   0 Batch 22275/28124   train_loss = 0.422  accuracy = 0.78125  auc = 0.8727272727272727\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:41:27.918252: Epoch   0 Batch 22300/28124   train_loss = 0.353  accuracy = 0.8125  auc = 0.6666666666666666\n",
      "2017-12-30T20:41:29.099634: Epoch   0 Batch 22325/28124   train_loss = 0.393  accuracy = 0.78125  auc = 0.875\n",
      "2017-12-30T20:41:30.227531: Epoch   0 Batch 22350/28124   train_loss = 0.450  accuracy = 0.78125  auc = 0.6962962962962963\n",
      "2017-12-30T20:41:31.367336: Epoch   0 Batch 22375/28124   train_loss = 0.392  accuracy = 0.8125  auc = 0.7777777777777778\n",
      "2017-12-30T20:41:32.485093: Epoch   0 Batch 22400/28124   train_loss = 0.511  accuracy = 0.75  auc = 0.8125\n",
      "2017-12-30T20:41:33.629599: Epoch   0 Batch 22425/28124   train_loss = 0.538  accuracy = 0.75  auc = 0.8259109311740891\n",
      "2017-12-30T20:41:34.741485: Epoch   0 Batch 22450/28124   train_loss = 0.319  accuracy = 0.875  auc = 0.8518518518518519\n",
      "2017-12-30T20:41:35.878986: Epoch   0 Batch 22475/28124   train_loss = 0.399  accuracy = 0.84375  auc = 0.7703703703703705\n",
      "2017-12-30T20:41:37.013244: Epoch   0 Batch 22500/28124   train_loss = 0.318  accuracy = 0.875  auc = 0.8782051282051282\n",
      "2017-12-30T20:41:38.218031: Epoch   0 Batch 22525/28124   train_loss = 0.332  accuracy = 0.875  auc = 0.8846153846153846\n",
      "2017-12-30T20:41:39.356416: Epoch   0 Batch 22550/28124   train_loss = 0.373  accuracy = 0.84375  auc = 0.8269230769230769\n",
      "2017-12-30T20:41:40.489420: Epoch   0 Batch 22575/28124   train_loss = 0.549  accuracy = 0.78125  auc = 0.7958333333333333\n",
      "2017-12-30T20:41:41.609739: Epoch   0 Batch 22600/28124   train_loss = 0.327  accuracy = 0.875  auc = 0.9696969696969696\n",
      "2017-12-30T20:41:42.744935: Epoch   0 Batch 22625/28124   train_loss = 0.472  accuracy = 0.8125  auc = 0.7111111111111112\n",
      "2017-12-30T20:41:43.894008: Epoch   0 Batch 22650/28124   train_loss = 0.487  accuracy = 0.8125  auc = 0.7135416666666666\n",
      "2017-12-30T20:41:45.056967: Epoch   0 Batch 22675/28124   train_loss = 0.358  accuracy = 0.90625  auc = 0.8782051282051283\n",
      "2017-12-30T20:41:46.196704: Epoch   0 Batch 22700/28124   train_loss = 0.494  accuracy = 0.75  auc = 0.71875\n",
      "2017-12-30T20:41:47.344457: Epoch   0 Batch 22725/28124   train_loss = 0.420  accuracy = 0.78125  auc = 0.8888888888888888\n",
      "2017-12-30T20:41:48.477980: Epoch   0 Batch 22750/28124   train_loss = 0.662  accuracy = 0.625  auc = 0.7764705882352941\n",
      "2017-12-30T20:41:49.611535: Epoch   0 Batch 22775/28124   train_loss = 0.348  accuracy = 0.8125  auc = 0.891025641025641\n",
      "2017-12-30T20:41:50.758242: Epoch   0 Batch 22800/28124   train_loss = 0.531  accuracy = 0.71875  auc = 0.8178137651821863\n",
      "2017-12-30T20:41:51.889433: Epoch   0 Batch 22825/28124   train_loss = 0.535  accuracy = 0.8125  auc = 0.7772727272727272\n",
      "2017-12-30T20:41:53.097872: Epoch   0 Batch 22850/28124   train_loss = 0.428  accuracy = 0.8125  auc = 0.8405797101449275\n",
      "2017-12-30T20:41:54.253807: Epoch   0 Batch 22875/28124   train_loss = 0.366  accuracy = 0.84375  auc = 0.9114583333333334\n",
      "2017-12-30T20:41:55.411197: Epoch   0 Batch 22900/28124   train_loss = 0.330  accuracy = 0.84375  auc = 0.8839285714285714\n",
      "2017-12-30T20:41:56.573193: Epoch   0 Batch 22925/28124   train_loss = 0.426  accuracy = 0.8125  auc = 0.725925925925926\n",
      "2017-12-30T20:41:57.717224: Epoch   0 Batch 22950/28124   train_loss = 0.500  accuracy = 0.75  auc = 0.85\n",
      "2017-12-30T20:41:58.859574: Epoch   0 Batch 22975/28124   train_loss = 0.487  accuracy = 0.75  auc = 0.7863636363636364\n",
      "2017-12-30T20:42:00.021771: Epoch   0 Batch 23000/28124   train_loss = 0.366  accuracy = 0.78125  auc = 0.8666666666666667\n",
      "2017-12-30T20:42:01.202499: Epoch   0 Batch 23025/28124   train_loss = 0.523  accuracy = 0.71875  auc = 0.8916666666666667\n",
      "2017-12-30T20:42:02.356659: Epoch   0 Batch 23050/28124   train_loss = 0.493  accuracy = 0.78125  auc = 0.6153846153846154\n",
      "2017-12-30T20:42:03.517015: Epoch   0 Batch 23075/28124   train_loss = 0.392  accuracy = 0.84375  auc = 0.8514285714285714\n",
      "2017-12-30T20:42:04.661914: Epoch   0 Batch 23100/28124   train_loss = 0.321  accuracy = 0.9375  auc = 0.8653846153846154\n",
      "2017-12-30T20:42:05.803675: Epoch   0 Batch 23125/28124   train_loss = 0.408  accuracy = 0.78125  auc = 0.8489583333333334\n",
      "2017-12-30T20:42:06.958269: Epoch   0 Batch 23150/28124   train_loss = 0.431  accuracy = 0.84375  auc = 0.6370370370370371\n",
      "2017-12-30T20:42:08.132287: Epoch   0 Batch 23175/28124   train_loss = 0.591  accuracy = 0.71875  auc = 0.6570048309178744\n",
      "2017-12-30T20:42:09.270727: Epoch   0 Batch 23200/28124   train_loss = 0.455  accuracy = 0.84375  auc = 0.7428571428571429\n",
      "2017-12-30T20:42:10.363660: Epoch   0 Batch 23225/28124   train_loss = 0.529  accuracy = 0.75  auc = 0.7791666666666667\n",
      "2017-12-30T20:42:11.503628: Epoch   0 Batch 23250/28124   train_loss = 0.444  accuracy = 0.8125  auc = 0.782051282051282\n",
      "2017-12-30T20:42:12.626560: Epoch   0 Batch 23275/28124   train_loss = 0.404  accuracy = 0.78125  auc = 0.8685714285714285\n",
      "2017-12-30T20:42:13.768228: Epoch   0 Batch 23300/28124   train_loss = 0.455  accuracy = 0.75  auc = 0.6785714285714286\n",
      "2017-12-30T20:42:14.925083: Epoch   0 Batch 23325/28124   train_loss = 0.488  accuracy = 0.75  auc = 0.7777777777777778\n",
      "2017-12-30T20:42:16.052163: Epoch   0 Batch 23350/28124   train_loss = 0.308  accuracy = 0.875  auc = 0.9855072463768115\n",
      "2017-12-30T20:42:17.228445: Epoch   0 Batch 23375/28124   train_loss = 0.385  accuracy = 0.8125  auc = 0.8074074074074074\n",
      "2017-12-30T20:42:18.369316: Epoch   0 Batch 23400/28124   train_loss = 0.405  accuracy = 0.8125  auc = 0.7885714285714286\n",
      "2017-12-30T20:42:19.500993: Epoch   0 Batch 23425/28124   train_loss = 0.469  accuracy = 0.75  auc = 0.8260869565217391\n",
      "2017-12-30T20:42:20.635527: Epoch   0 Batch 23450/28124   train_loss = 0.261  accuracy = 0.90625  auc = 0.9743589743589743\n",
      "2017-12-30T20:42:21.752338: Epoch   0 Batch 23475/28124   train_loss = 0.390  accuracy = 0.75  auc = 0.8457142857142856\n",
      "2017-12-30T20:42:22.907670: Epoch   0 Batch 23500/28124   train_loss = 0.545  accuracy = 0.75  auc = 0.7772727272727272\n",
      "2017-12-30T20:42:24.100346: Epoch   0 Batch 23525/28124   train_loss = 0.579  accuracy = 0.6875  auc = 0.6908212560386473\n",
      "2017-12-30T20:42:25.259451: Epoch   0 Batch 23550/28124   train_loss = 0.283  accuracy = 1.0  auc = 1.0\n",
      "2017-12-30T20:42:26.395548: Epoch   0 Batch 23575/28124   train_loss = 0.594  accuracy = 0.65625  auc = 0.7333333333333334\n",
      "2017-12-30T20:42:27.510843: Epoch   0 Batch 23600/28124   train_loss = 0.461  accuracy = 0.78125  auc = 0.7314285714285713\n",
      "2017-12-30T20:42:28.636325: Epoch   0 Batch 23625/28124   train_loss = 0.522  accuracy = 0.6875  auc = 0.7792207792207793\n",
      "2017-12-30T20:42:29.773423: Epoch   0 Batch 23650/28124   train_loss = 0.546  accuracy = 0.65625  auc = 0.8138528138528138\n",
      "2017-12-30T20:42:30.947099: Epoch   0 Batch 23675/28124   train_loss = 0.455  accuracy = 0.8125  auc = 0.7371428571428571\n",
      "2017-12-30T20:42:32.125744: Epoch   0 Batch 23700/28124   train_loss = 0.422  accuracy = 0.8125  auc = 0.8961038961038961\n",
      "2017-12-30T20:42:33.261826: Epoch   0 Batch 23725/28124   train_loss = 0.385  accuracy = 0.84375  auc = 0.8076923076923077\n",
      "2017-12-30T20:42:34.396581: Epoch   0 Batch 23750/28124   train_loss = 0.455  accuracy = 0.78125  auc = 0.7428571428571428\n",
      "2017-12-30T20:42:35.550631: Epoch   0 Batch 23775/28124   train_loss = 0.457  accuracy = 0.78125  auc = 0.7142857142857142\n",
      "2017-12-30T20:42:36.701828: Epoch   0 Batch 23800/28124   train_loss = 0.567  accuracy = 0.6875  auc = 0.8137651821862348\n",
      "2017-12-30T20:42:37.879526: Epoch   0 Batch 23825/28124   train_loss = 0.448  accuracy = 0.8125  auc = 0.8136363636363636\n",
      "2017-12-30T20:42:39.028622: Epoch   0 Batch 23850/28124   train_loss = 0.554  accuracy = 0.6875  auc = 0.7922077922077921\n",
      "2017-12-30T20:42:40.225522: Epoch   0 Batch 23875/28124   train_loss = 0.411  accuracy = 0.78125  auc = 0.8285714285714285\n",
      "2017-12-30T20:42:41.386344: Epoch   0 Batch 23900/28124   train_loss = 0.261  accuracy = 0.9375  auc = 0.9555555555555556\n",
      "2017-12-30T20:42:42.531406: Epoch   0 Batch 23925/28124   train_loss = 0.539  accuracy = 0.71875  auc = 0.6628571428571428\n",
      "2017-12-30T20:42:43.662257: Epoch   0 Batch 23950/28124   train_loss = 0.340  accuracy = 0.90625  auc = 0.8074074074074075\n",
      "2017-12-30T20:42:44.808542: Epoch   0 Batch 23975/28124   train_loss = 0.443  accuracy = 0.84375  auc = 0.7111111111111111\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:42:45.951569: Epoch   0 Batch 24000/28124   train_loss = 0.308  accuracy = 0.96875  auc = 0.9732142857142857\n",
      "2017-12-30T20:42:47.149789: Epoch   0 Batch 24025/28124   train_loss = 0.568  accuracy = 0.75  auc = 0.7445887445887446\n",
      "2017-12-30T20:42:48.288072: Epoch   0 Batch 24050/28124   train_loss = 0.359  accuracy = 0.84375  auc = 0.8697916666666666\n",
      "2017-12-30T20:42:49.433862: Epoch   0 Batch 24075/28124   train_loss = 0.360  accuracy = 0.875  auc = 0.8800000000000001\n",
      "2017-12-30T20:42:50.589658: Epoch   0 Batch 24100/28124   train_loss = 0.300  accuracy = 0.875  auc = 0.9542857142857142\n",
      "2017-12-30T20:42:51.787864: Epoch   0 Batch 24125/28124   train_loss = 0.366  accuracy = 0.8125  auc = 0.8685714285714285\n",
      "2017-12-30T20:42:52.959161: Epoch   0 Batch 24150/28124   train_loss = 0.498  accuracy = 0.75  auc = 0.8\n",
      "2017-12-30T20:42:54.066450: Epoch   0 Batch 24175/28124   train_loss = 0.447  accuracy = 0.8125  auc = 0.7916666666666667\n",
      "2017-12-30T20:42:55.223692: Epoch   0 Batch 24200/28124   train_loss = 0.493  accuracy = 0.78125  auc = 0.8291666666666667\n",
      "2017-12-30T20:42:56.377065: Epoch   0 Batch 24225/28124   train_loss = 0.368  accuracy = 0.875  auc = 0.8653846153846153\n",
      "2017-12-30T20:42:57.514581: Epoch   0 Batch 24250/28124   train_loss = 0.256  accuracy = 0.9375  auc = 0.9555555555555556\n",
      "2017-12-30T20:42:58.656419: Epoch   0 Batch 24275/28124   train_loss = 0.574  accuracy = 0.5625  auc = 0.6908212560386473\n",
      "2017-12-30T20:42:59.785296: Epoch   0 Batch 24300/28124   train_loss = 0.474  accuracy = 0.71875  auc = 0.7971014492753623\n",
      "2017-12-30T20:43:00.924679: Epoch   0 Batch 24325/28124   train_loss = 0.303  accuracy = 0.90625  auc = 0.9615384615384616\n",
      "2017-12-30T20:43:02.105410: Epoch   0 Batch 24350/28124   train_loss = 0.488  accuracy = 0.8125  auc = 0.7922705314009661\n",
      "2017-12-30T20:43:03.250439: Epoch   0 Batch 24375/28124   train_loss = 0.513  accuracy = 0.71875  auc = 0.7632850241545893\n",
      "2017-12-30T20:43:04.373736: Epoch   0 Batch 24400/28124   train_loss = 0.353  accuracy = 0.8125  auc = 0.8974358974358975\n",
      "2017-12-30T20:43:05.511269: Epoch   0 Batch 24425/28124   train_loss = 0.487  accuracy = 0.75  auc = 0.75\n",
      "2017-12-30T20:43:06.658023: Epoch   0 Batch 24450/28124   train_loss = 0.403  accuracy = 0.84375  auc = 0.8863636363636364\n",
      "2017-12-30T20:43:07.803149: Epoch   0 Batch 24475/28124   train_loss = 0.312  accuracy = 0.90625  auc = 0.8814814814814814\n",
      "2017-12-30T20:43:08.966455: Epoch   0 Batch 24500/28124   train_loss = 0.460  accuracy = 0.71875  auc = 0.7756410256410257\n",
      "2017-12-30T20:43:10.156324: Epoch   0 Batch 24525/28124   train_loss = 0.709  accuracy = 0.59375  auc = 0.74609375\n",
      "2017-12-30T20:43:11.299344: Epoch   0 Batch 24550/28124   train_loss = 0.395  accuracy = 0.84375  auc = 0.8802083333333333\n",
      "2017-12-30T20:43:12.424634: Epoch   0 Batch 24575/28124   train_loss = 0.331  accuracy = 0.90625  auc = 0.8974358974358975\n",
      "2017-12-30T20:43:13.578267: Epoch   0 Batch 24600/28124   train_loss = 0.384  accuracy = 0.8125  auc = 0.9727272727272728\n",
      "2017-12-30T20:43:14.722248: Epoch   0 Batch 24625/28124   train_loss = 0.450  accuracy = 0.78125  auc = 0.8212560386473429\n",
      "2017-12-30T20:43:15.867001: Epoch   0 Batch 24650/28124   train_loss = 0.380  accuracy = 0.875  auc = 0.8599033816425121\n",
      "2017-12-30T20:43:17.050014: Epoch   0 Batch 24675/28124   train_loss = 0.602  accuracy = 0.65625  auc = 0.7090909090909091\n",
      "2017-12-30T20:43:18.235050: Epoch   0 Batch 24700/28124   train_loss = 0.317  accuracy = 0.875  auc = 0.9333333333333333\n",
      "2017-12-30T20:43:19.378672: Epoch   0 Batch 24725/28124   train_loss = 0.315  accuracy = 0.875  auc = 0.8846153846153846\n",
      "2017-12-30T20:43:20.532843: Epoch   0 Batch 24750/28124   train_loss = 0.430  accuracy = 0.71875  auc = 0.8840579710144927\n",
      "2017-12-30T20:43:21.667677: Epoch   0 Batch 24775/28124   train_loss = 0.626  accuracy = 0.71875  auc = 0.7619047619047619\n",
      "2017-12-30T20:43:22.815739: Epoch   0 Batch 24800/28124   train_loss = 0.378  accuracy = 0.8125  auc = 0.8205128205128206\n",
      "2017-12-30T20:43:23.966208: Epoch   0 Batch 24825/28124   train_loss = 0.316  accuracy = 0.875  auc = 0.8914285714285715\n",
      "2017-12-30T20:43:25.151056: Epoch   0 Batch 24850/28124   train_loss = 0.509  accuracy = 0.71875  auc = 0.6685714285714286\n",
      "2017-12-30T20:43:26.298572: Epoch   0 Batch 24875/28124   train_loss = 0.430  accuracy = 0.8125  auc = 0.8115942028985508\n",
      "2017-12-30T20:43:27.447426: Epoch   0 Batch 24900/28124   train_loss = 0.423  accuracy = 0.8125  auc = 0.9177489177489178\n",
      "2017-12-30T20:43:28.585921: Epoch   0 Batch 24925/28124   train_loss = 0.547  accuracy = 0.75  auc = 0.854251012145749\n",
      "2017-12-30T20:43:29.723105: Epoch   0 Batch 24950/28124   train_loss = 0.383  accuracy = 0.78125  auc = 0.8541666666666666\n",
      "2017-12-30T20:43:30.885309: Epoch   0 Batch 24975/28124   train_loss = 0.530  accuracy = 0.75  auc = 0.5535714285714286\n",
      "2017-12-30T20:43:32.063737: Epoch   0 Batch 25000/28124   train_loss = 0.331  accuracy = 0.875  auc = 0.8717948717948718\n",
      "2017-12-30T20:43:33.232442: Epoch   0 Batch 25025/28124   train_loss = 0.387  accuracy = 0.8125  auc = 0.8802083333333334\n",
      "2017-12-30T20:43:34.382951: Epoch   0 Batch 25050/28124   train_loss = 0.431  accuracy = 0.78125  auc = 0.8141025641025641\n",
      "2017-12-30T20:43:35.506399: Epoch   0 Batch 25075/28124   train_loss = 0.558  accuracy = 0.78125  auc = 0.7227272727272727\n",
      "2017-12-30T20:43:36.636989: Epoch   0 Batch 25100/28124   train_loss = 0.386  accuracy = 0.84375  auc = 0.725925925925926\n",
      "2017-12-30T20:43:37.784150: Epoch   0 Batch 25125/28124   train_loss = 0.483  accuracy = 0.78125  auc = 0.7971014492753623\n",
      "2017-12-30T20:43:38.950572: Epoch   0 Batch 25150/28124   train_loss = 0.376  accuracy = 0.78125  auc = 0.8888888888888888\n",
      "2017-12-30T20:43:40.174849: Epoch   0 Batch 25175/28124   train_loss = 0.519  accuracy = 0.8125  auc = 0.7135416666666666\n",
      "2017-12-30T20:43:41.379715: Epoch   0 Batch 25200/28124   train_loss = 0.529  accuracy = 0.78125  auc = 0.6956521739130435\n",
      "2017-12-30T20:43:42.559380: Epoch   0 Batch 25225/28124   train_loss = 0.540  accuracy = 0.75  auc = 0.8531746031746033\n",
      "2017-12-30T20:43:43.708034: Epoch   0 Batch 25250/28124   train_loss = 0.419  accuracy = 0.75  auc = 0.8171428571428572\n",
      "2017-12-30T20:43:44.873053: Epoch   0 Batch 25275/28124   train_loss = 0.425  accuracy = 0.71875  auc = 0.8171428571428572\n",
      "2017-12-30T20:43:46.002743: Epoch   0 Batch 25300/28124   train_loss = 0.611  accuracy = 0.65625  auc = 0.6818181818181819\n",
      "2017-12-30T20:43:47.175220: Epoch   0 Batch 25325/28124   train_loss = 0.707  accuracy = 0.625  auc = 0.6045454545454546\n",
      "2017-12-30T20:43:48.321560: Epoch   0 Batch 25350/28124   train_loss = 0.645  accuracy = 0.625  auc = 0.7333333333333334\n",
      "2017-12-30T20:43:49.458543: Epoch   0 Batch 25375/28124   train_loss = 0.450  accuracy = 0.65625  auc = 0.7604166666666667\n",
      "2017-12-30T20:43:50.605160: Epoch   0 Batch 25400/28124   train_loss = 0.540  accuracy = 0.6875  auc = 0.8137651821862348\n",
      "2017-12-30T20:43:51.745459: Epoch   0 Batch 25425/28124   train_loss = 0.434  accuracy = 0.78125  auc = 0.625\n",
      "2017-12-30T20:43:52.893496: Epoch   0 Batch 25450/28124   train_loss = 0.561  accuracy = 0.75  auc = 0.8373015873015873\n",
      "2017-12-30T20:43:54.082440: Epoch   0 Batch 25475/28124   train_loss = 0.427  accuracy = 0.84375  auc = 0.8740740740740741\n",
      "2017-12-30T20:43:55.234994: Epoch   0 Batch 25500/28124   train_loss = 0.480  accuracy = 0.8125  auc = 0.8658008658008658\n",
      "2017-12-30T20:43:56.408824: Epoch   0 Batch 25525/28124   train_loss = 0.419  accuracy = 0.84375  auc = 0.7243589743589743\n",
      "2017-12-30T20:43:57.564706: Epoch   0 Batch 25550/28124   train_loss = 0.418  accuracy = 0.8125  auc = 0.7692307692307692\n",
      "2017-12-30T20:43:58.717671: Epoch   0 Batch 25575/28124   train_loss = 0.336  accuracy = 0.90625  auc = 0.8392857142857143\n",
      "2017-12-30T20:43:59.876675: Epoch   0 Batch 25600/28124   train_loss = 0.677  accuracy = 0.65625  auc = 0.6147186147186148\n",
      "2017-12-30T20:44:01.058517: Epoch   0 Batch 25625/28124   train_loss = 0.383  accuracy = 0.84375  auc = 0.8514285714285714\n",
      "2017-12-30T20:44:02.247412: Epoch   0 Batch 25650/28124   train_loss = 0.623  accuracy = 0.625  auc = 0.7817460317460317\n",
      "2017-12-30T20:44:03.385242: Epoch   0 Batch 25675/28124   train_loss = 0.499  accuracy = 0.71875  auc = 0.7729468599033816\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:44:04.539584: Epoch   0 Batch 25700/28124   train_loss = 0.415  accuracy = 0.84375  auc = 0.859375\n",
      "2017-12-30T20:44:05.688604: Epoch   0 Batch 25725/28124   train_loss = 0.425  accuracy = 0.8125  auc = 0.8550724637681159\n",
      "2017-12-30T20:44:06.828571: Epoch   0 Batch 25750/28124   train_loss = 0.486  accuracy = 0.71875  auc = 0.8260869565217392\n",
      "2017-12-30T20:44:07.971484: Epoch   0 Batch 25775/28124   train_loss = 0.560  accuracy = 0.6875  auc = 0.740909090909091\n",
      "2017-12-30T20:44:09.169796: Epoch   0 Batch 25800/28124   train_loss = 0.560  accuracy = 0.71875  auc = 0.6714975845410628\n",
      "2017-12-30T20:44:10.321354: Epoch   0 Batch 25825/28124   train_loss = 0.339  accuracy = 0.875  auc = 0.875\n",
      "2017-12-30T20:44:11.454457: Epoch   0 Batch 25850/28124   train_loss = 0.449  accuracy = 0.84375  auc = 0.8454545454545455\n",
      "2017-12-30T20:44:12.596496: Epoch   0 Batch 25875/28124   train_loss = 0.451  accuracy = 0.78125  auc = 0.8454106280193237\n",
      "2017-12-30T20:44:13.749230: Epoch   0 Batch 25900/28124   train_loss = 0.358  accuracy = 0.84375  auc = 0.8717948717948718\n",
      "2017-12-30T20:44:14.917197: Epoch   0 Batch 25925/28124   train_loss = 0.574  accuracy = 0.75  auc = 0.6171428571428572\n",
      "2017-12-30T20:44:16.108428: Epoch   0 Batch 25950/28124   train_loss = 0.422  accuracy = 0.84375  auc = 0.8628571428571428\n",
      "2017-12-30T20:44:17.244485: Epoch   0 Batch 25975/28124   train_loss = 0.553  accuracy = 0.65625  auc = 0.75\n",
      "2017-12-30T20:44:18.381182: Epoch   0 Batch 26000/28124   train_loss = 0.425  accuracy = 0.875  auc = 0.7371794871794872\n",
      "2017-12-30T20:44:19.518584: Epoch   0 Batch 26025/28124   train_loss = 0.448  accuracy = 0.75  auc = 0.8405797101449275\n",
      "2017-12-30T20:44:20.641286: Epoch   0 Batch 26050/28124   train_loss = 0.342  accuracy = 0.875  auc = 0.9166666666666666\n",
      "2017-12-30T20:44:21.792500: Epoch   0 Batch 26075/28124   train_loss = 0.285  accuracy = 0.90625  auc = 0.9375\n",
      "2017-12-30T20:44:22.946833: Epoch   0 Batch 26100/28124   train_loss = 0.325  accuracy = 0.84375  auc = 0.8303571428571428\n",
      "2017-12-30T20:44:24.137297: Epoch   0 Batch 26125/28124   train_loss = 0.597  accuracy = 0.6875  auc = 0.7291666666666666\n",
      "2017-12-30T20:44:25.288146: Epoch   0 Batch 26150/28124   train_loss = 0.533  accuracy = 0.71875  auc = 0.8208333333333333\n",
      "2017-12-30T20:44:26.431740: Epoch   0 Batch 26175/28124   train_loss = 0.421  accuracy = 0.75  auc = 0.8260869565217391\n",
      "2017-12-30T20:44:27.581428: Epoch   0 Batch 26200/28124   train_loss = 0.361  accuracy = 0.875  auc = 0.9010416666666667\n",
      "2017-12-30T20:44:28.725989: Epoch   0 Batch 26225/28124   train_loss = 0.455  accuracy = 0.78125  auc = 0.882591093117409\n",
      "2017-12-30T20:44:29.865974: Epoch   0 Batch 26250/28124   train_loss = 0.610  accuracy = 0.75  auc = 0.6363636363636364\n",
      "2017-12-30T20:44:31.014439: Epoch   0 Batch 26275/28124   train_loss = 0.536  accuracy = 0.71875  auc = 0.7681818181818183\n",
      "2017-12-30T20:44:32.179961: Epoch   0 Batch 26300/28124   train_loss = 0.552  accuracy = 0.65625  auc = 0.7454545454545455\n",
      "2017-12-30T20:44:33.346552: Epoch   0 Batch 26325/28124   train_loss = 0.470  accuracy = 0.78125  auc = 0.7628205128205128\n",
      "2017-12-30T20:44:34.504377: Epoch   0 Batch 26350/28124   train_loss = 0.375  accuracy = 0.78125  auc = 0.8571428571428571\n",
      "2017-12-30T20:44:35.647802: Epoch   0 Batch 26375/28124   train_loss = 0.482  accuracy = 0.75  auc = 0.8181818181818181\n",
      "2017-12-30T20:44:36.796830: Epoch   0 Batch 26400/28124   train_loss = 0.462  accuracy = 0.75  auc = 0.8229166666666666\n",
      "2017-12-30T20:44:37.977701: Epoch   0 Batch 26425/28124   train_loss = 0.656  accuracy = 0.6875  auc = 0.7023809523809523\n",
      "2017-12-30T20:44:39.167254: Epoch   0 Batch 26450/28124   train_loss = 0.425  accuracy = 0.78125  auc = 0.7885714285714286\n",
      "2017-12-30T20:44:40.314438: Epoch   0 Batch 26475/28124   train_loss = 0.383  accuracy = 0.84375  auc = 0.8695652173913043\n",
      "2017-12-30T20:44:41.453130: Epoch   0 Batch 26500/28124   train_loss = 0.519  accuracy = 0.8125  auc = 0.7705627705627706\n",
      "2017-12-30T20:44:42.602551: Epoch   0 Batch 26525/28124   train_loss = 0.490  accuracy = 0.71875  auc = 0.6474358974358975\n",
      "2017-12-30T20:44:43.740535: Epoch   0 Batch 26550/28124   train_loss = 0.627  accuracy = 0.65625  auc = 0.7708333333333333\n",
      "2017-12-30T20:44:44.877509: Epoch   0 Batch 26575/28124   train_loss = 0.665  accuracy = 0.625  auc = 0.6875\n",
      "2017-12-30T20:44:46.026527: Epoch   0 Batch 26600/28124   train_loss = 0.392  accuracy = 0.8125  auc = 0.8645833333333333\n",
      "2017-12-30T20:44:47.208418: Epoch   0 Batch 26625/28124   train_loss = 0.449  accuracy = 0.8125  auc = 0.7657142857142857\n",
      "2017-12-30T20:44:48.360611: Epoch   0 Batch 26650/28124   train_loss = 0.364  accuracy = 0.84375  auc = 0.7703703703703704\n",
      "2017-12-30T20:44:49.499327: Epoch   0 Batch 26675/28124   train_loss = 0.366  accuracy = 0.84375  auc = 0.90625\n",
      "2017-12-30T20:44:50.661657: Epoch   0 Batch 26700/28124   train_loss = 0.304  accuracy = 0.90625  auc = 0.9038461538461539\n",
      "2017-12-30T20:44:51.847334: Epoch   0 Batch 26725/28124   train_loss = 0.415  accuracy = 0.8125  auc = 0.8743961352657005\n",
      "2017-12-30T20:44:53.015473: Epoch   0 Batch 26750/28124   train_loss = 0.680  accuracy = 0.71875  auc = 0.4895833333333333\n",
      "2017-12-30T20:44:54.209468: Epoch   0 Batch 26775/28124   train_loss = 0.377  accuracy = 0.875  auc = 0.84375\n",
      "2017-12-30T20:44:55.361453: Epoch   0 Batch 26800/28124   train_loss = 0.397  accuracy = 0.8125  auc = 0.7771428571428571\n",
      "2017-12-30T20:44:56.533230: Epoch   0 Batch 26825/28124   train_loss = 0.434  accuracy = 0.78125  auc = 0.8385416666666666\n",
      "2017-12-30T20:44:57.669699: Epoch   0 Batch 26850/28124   train_loss = 0.502  accuracy = 0.78125  auc = 0.8095238095238095\n",
      "2017-12-30T20:44:58.813928: Epoch   0 Batch 26875/28124   train_loss = 0.623  accuracy = 0.65625  auc = 0.6570048309178744\n",
      "2017-12-30T20:44:59.971486: Epoch   0 Batch 26900/28124   train_loss = 0.480  accuracy = 0.78125  auc = 0.78743961352657\n",
      "2017-12-30T20:45:01.149114: Epoch   0 Batch 26925/28124   train_loss = 0.341  accuracy = 0.96875  auc = 0.9111111111111112\n",
      "2017-12-30T20:45:02.295294: Epoch   0 Batch 26950/28124   train_loss = 0.546  accuracy = 0.71875  auc = 0.7772727272727273\n",
      "2017-12-30T20:45:03.436270: Epoch   0 Batch 26975/28124   train_loss = 0.336  accuracy = 0.875  auc = 0.851851851851852\n",
      "2017-12-30T20:45:04.570212: Epoch   0 Batch 27000/28124   train_loss = 0.499  accuracy = 0.8125  auc = 0.8250000000000001\n",
      "2017-12-30T20:45:05.712048: Epoch   0 Batch 27025/28124   train_loss = 0.350  accuracy = 0.84375  auc = 0.9314285714285714\n",
      "2017-12-30T20:45:06.856306: Epoch   0 Batch 27050/28124   train_loss = 0.420  accuracy = 0.84375  auc = 0.7692307692307693\n",
      "2017-12-30T20:45:08.010162: Epoch   0 Batch 27075/28124   train_loss = 0.495  accuracy = 0.71875  auc = 0.8441558441558442\n",
      "2017-12-30T20:45:09.192229: Epoch   0 Batch 27100/28124   train_loss = 0.416  accuracy = 0.84375  auc = 0.8125\n",
      "2017-12-30T20:45:10.347514: Epoch   0 Batch 27125/28124   train_loss = 0.701  accuracy = 0.75  auc = 0.6666666666666666\n",
      "2017-12-30T20:45:11.468724: Epoch   0 Batch 27150/28124   train_loss = 0.702  accuracy = 0.625  auc = 0.6963562753036439\n",
      "2017-12-30T20:45:12.612353: Epoch   0 Batch 27175/28124   train_loss = 0.565  accuracy = 0.65625  auc = 0.774891774891775\n",
      "2017-12-30T20:45:13.754696: Epoch   0 Batch 27200/28124   train_loss = 0.327  accuracy = 0.875  auc = 0.9199999999999999\n",
      "2017-12-30T20:45:14.921309: Epoch   0 Batch 27225/28124   train_loss = 0.483  accuracy = 0.78125  auc = 0.8136363636363637\n",
      "2017-12-30T20:45:16.120010: Epoch   0 Batch 27250/28124   train_loss = 0.387  accuracy = 0.84375  auc = 0.8357487922705314\n",
      "2017-12-30T20:45:17.263668: Epoch   0 Batch 27275/28124   train_loss = 0.436  accuracy = 0.8125  auc = 0.855072463768116\n",
      "2017-12-30T20:45:18.396758: Epoch   0 Batch 27300/28124   train_loss = 0.304  accuracy = 0.875  auc = 0.9107142857142857\n",
      "2017-12-30T20:45:19.542860: Epoch   0 Batch 27325/28124   train_loss = 0.524  accuracy = 0.8125  auc = 0.748792270531401\n",
      "2017-12-30T20:45:20.675059: Epoch   0 Batch 27350/28124   train_loss = 0.317  accuracy = 0.875  auc = 0.8962962962962964\n",
      "2017-12-30T20:45:21.812427: Epoch   0 Batch 27375/28124   train_loss = 0.437  accuracy = 0.78125  auc = 0.7371794871794872\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:45:22.954815: Epoch   0 Batch 27400/28124   train_loss = 0.570  accuracy = 0.71875  auc = 0.7619047619047619\n",
      "2017-12-30T20:45:24.144342: Epoch   0 Batch 27425/28124   train_loss = 0.319  accuracy = 0.875  auc = 0.8660714285714286\n",
      "2017-12-30T20:45:25.311803: Epoch   0 Batch 27450/28124   train_loss = 0.399  accuracy = 0.75  auc = 0.9291666666666667\n",
      "2017-12-30T20:45:26.465085: Epoch   0 Batch 27475/28124   train_loss = 0.531  accuracy = 0.84375  auc = 0.6057142857142856\n",
      "2017-12-30T20:45:27.608333: Epoch   0 Batch 27500/28124   train_loss = 0.806  accuracy = 0.5625  auc = 0.5791666666666667\n",
      "2017-12-30T20:45:28.743698: Epoch   0 Batch 27525/28124   train_loss = 0.429  accuracy = 0.8125  auc = 0.8171428571428572\n",
      "2017-12-30T20:45:29.878110: Epoch   0 Batch 27550/28124   train_loss = 0.294  accuracy = 0.90625  auc = 0.911111111111111\n",
      "2017-12-30T20:45:31.026814: Epoch   0 Batch 27575/28124   train_loss = 0.550  accuracy = 0.75  auc = 0.6302083333333334\n",
      "2017-12-30T20:45:32.196399: Epoch   0 Batch 27600/28124   train_loss = 0.516  accuracy = 0.6875  auc = 0.7318181818181818\n",
      "2017-12-30T20:45:33.364327: Epoch   0 Batch 27625/28124   train_loss = 0.489  accuracy = 0.78125  auc = 0.8704453441295547\n",
      "2017-12-30T20:45:34.509029: Epoch   0 Batch 27650/28124   train_loss = 0.423  accuracy = 0.8125  auc = 0.8541666666666667\n",
      "2017-12-30T20:45:35.644044: Epoch   0 Batch 27675/28124   train_loss = 0.506  accuracy = 0.71875  auc = 0.7954545454545454\n",
      "2017-12-30T20:45:36.775956: Epoch   0 Batch 27700/28124   train_loss = 0.292  accuracy = 0.84375  auc = 0.9806763285024154\n",
      "2017-12-30T20:45:37.945682: Epoch   0 Batch 27725/28124   train_loss = 0.544  accuracy = 0.78125  auc = 0.7004830917874396\n",
      "2017-12-30T20:45:39.144751: Epoch   0 Batch 27750/28124   train_loss = 0.402  accuracy = 0.84375  auc = 0.8342857142857143\n",
      "2017-12-30T20:45:40.292374: Epoch   0 Batch 27775/28124   train_loss = 0.465  accuracy = 0.8125  auc = 0.8528138528138528\n",
      "2017-12-30T20:45:41.455237: Epoch   0 Batch 27800/28124   train_loss = 0.348  accuracy = 0.84375  auc = 0.875\n",
      "2017-12-30T20:45:42.600707: Epoch   0 Batch 27825/28124   train_loss = 0.404  accuracy = 0.8125  auc = 0.8114285714285714\n",
      "2017-12-30T20:45:43.736799: Epoch   0 Batch 27850/28124   train_loss = 0.377  accuracy = 0.875  auc = 0.828125\n",
      "2017-12-30T20:45:44.882565: Epoch   0 Batch 27875/28124   train_loss = 0.518  accuracy = 0.65625  auc = 0.8458333333333333\n",
      "2017-12-30T20:45:46.051230: Epoch   0 Batch 27900/28124   train_loss = 0.408  accuracy = 0.8125  auc = 0.8545454545454545\n",
      "2017-12-30T20:45:47.237211: Epoch   0 Batch 27925/28124   train_loss = 0.412  accuracy = 0.78125  auc = 0.8333333333333333\n",
      "2017-12-30T20:45:48.380909: Epoch   0 Batch 27950/28124   train_loss = 0.392  accuracy = 0.75  auc = 0.8697916666666667\n",
      "2017-12-30T20:45:49.530432: Epoch   0 Batch 27975/28124   train_loss = 0.443  accuracy = 0.75  auc = 0.883116883116883\n",
      "2017-12-30T20:45:50.690605: Epoch   0 Batch 28000/28124   train_loss = 0.228  accuracy = 0.96875  auc = 0.9771428571428571\n",
      "2017-12-30T20:45:51.846185: Epoch   0 Batch 28025/28124   train_loss = 0.400  accuracy = 0.8125  auc = 0.8541666666666666\n",
      "2017-12-30T20:45:52.990507: Epoch   0 Batch 28050/28124   train_loss = 0.470  accuracy = 0.84375  auc = 0.8\n",
      "2017-12-30T20:45:54.177954: Epoch   0 Batch 28075/28124   train_loss = 0.373  accuracy = 0.875  auc = 0.8906249999999999\n",
      "2017-12-30T20:45:55.307479: Epoch   0 Batch 28100/28124   train_loss = 0.387  accuracy = 0.84375  auc = 0.6964285714285714\n",
      "2017-12-30T20:45:56.408594: Epoch   0 Batch    0/3125   test_loss = 0.621  accuracy = 0.75  auc = 0.5520833333333334\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.77      0.96      0.85        24\n",
      "        1.0       0.50      0.12      0.20         8\n",
      "\n",
      "avg / total       0.70      0.75      0.69        32\n",
      "\n",
      "2017-12-30T20:45:56.508973: Epoch   0 Batch   25/3125   test_loss = 0.491  accuracy = 0.75  auc = 0.6602564102564104\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.82      0.88      0.85        26\n",
      "        1.0       0.25      0.17      0.20         6\n",
      "\n",
      "avg / total       0.71      0.75      0.73        32\n",
      "\n",
      "2017-12-30T20:45:56.604274: Epoch   0 Batch   50/3125   test_loss = 0.482  accuracy = 0.75  auc = 0.6962962962962964\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.88      0.81      0.85        27\n",
      "        1.0       0.29      0.40      0.33         5\n",
      "\n",
      "avg / total       0.79      0.75      0.77        32\n",
      "\n",
      "2017-12-30T20:45:56.708047: Epoch   0 Batch   75/3125   test_loss = 0.423  accuracy = 0.8125  auc = 0.782051282051282\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.92      0.89        26\n",
      "        1.0       0.50      0.33      0.40         6\n",
      "\n",
      "avg / total       0.79      0.81      0.80        32\n",
      "\n",
      "2017-12-30T20:45:56.804918: Epoch   0 Batch  100/3125   test_loss = 0.504  accuracy = 0.75  auc = 0.6814814814814816\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.88      0.81      0.85        27\n",
      "        1.0       0.29      0.40      0.33         5\n",
      "\n",
      "avg / total       0.79      0.75      0.77        32\n",
      "\n",
      "2017-12-30T20:45:56.905435: Epoch   0 Batch  125/3125   test_loss = 0.547  accuracy = 0.75  auc = 0.7294685990338164\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.80      0.87      0.83        23\n",
      "        1.0       0.57      0.44      0.50         9\n",
      "\n",
      "avg / total       0.74      0.75      0.74        32\n",
      "\n",
      "2017-12-30T20:45:57.011853: Epoch   0 Batch  150/3125   test_loss = 0.621  accuracy = 0.65625  auc = 0.6231884057971014\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.75      0.78      0.77        23\n",
      "        1.0       0.38      0.33      0.35         9\n",
      "\n",
      "avg / total       0.64      0.66      0.65        32\n",
      "\n",
      "2017-12-30T20:45:57.178003: Epoch   0 Batch  175/3125   test_loss = 0.355  accuracy = 0.875  auc = 0.8269230769230769\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.89      0.96      0.93        26\n",
      "        1.0       0.75      0.50      0.60         6\n",
      "\n",
      "avg / total       0.87      0.88      0.86        32\n",
      "\n",
      "2017-12-30T20:45:57.280622: Epoch   0 Batch  200/3125   test_loss = 0.514  accuracy = 0.78125  auc = 0.8375000000000001\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.76      0.95      0.84        20\n",
      "        1.0       0.86      0.50      0.63        12\n",
      "\n",
      "avg / total       0.80      0.78      0.76        32\n",
      "\n",
      "2017-12-30T20:45:57.376627: Epoch   0 Batch  225/3125   test_loss = 0.287  accuracy = 0.90625  auc = 0.8160919540229885\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.91      1.00      0.95        29\n",
      "        1.0       0.00      0.00      0.00         3\n",
      "\n",
      "avg / total       0.82      0.91      0.86        32\n",
      "\n",
      "2017-12-30T20:45:57.473180: Epoch   0 Batch  250/3125   test_loss = 0.548  accuracy = 0.71875  auc = 0.8461538461538461\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.68      1.00      0.81        19\n",
      "        1.0       1.00      0.31      0.47        13\n",
      "\n",
      "avg / total       0.81      0.72      0.67        32\n",
      "\n",
      "2017-12-30T20:45:57.577462: Epoch   0 Batch  275/3125   test_loss = 0.522  accuracy = 0.71875  auc = 0.7391304347826086\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.75      0.91      0.82        23\n",
      "        1.0       0.50      0.22      0.31         9\n",
      "\n",
      "avg / total       0.68      0.72      0.68        32\n",
      "\n",
      "2017-12-30T20:45:57.675876: Epoch   0 Batch  300/3125   test_loss = 0.508  accuracy = 0.75  auc = 0.6742857142857143\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.87      0.80      0.83        25\n",
      "        1.0       0.44      0.57      0.50         7\n",
      "\n",
      "avg / total       0.78      0.75      0.76        32\n",
      "\n",
      "2017-12-30T20:45:57.772655: Epoch   0 Batch  325/3125   test_loss = 0.383  accuracy = 0.84375  auc = 0.7884615384615385\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.89      0.92      0.91        26\n",
      "        1.0       0.60      0.50      0.55         6\n",
      "\n",
      "avg / total       0.83      0.84      0.84        32\n",
      "\n",
      "2017-12-30T20:45:57.878316: Epoch   0 Batch  350/3125   test_loss = 0.504  accuracy = 0.8125  auc = 0.7536231884057971\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.79      1.00      0.88        23\n",
      "        1.0       1.00      0.33      0.50         9\n",
      "\n",
      "avg / total       0.85      0.81      0.78        32\n",
      "\n",
      "2017-12-30T20:45:57.977488: Epoch   0 Batch  375/3125   test_loss = 0.681  accuracy = 0.625  auc = 0.6103896103896105\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.66      0.90      0.76        21\n",
      "        1.0       0.33      0.09      0.14        11\n",
      "\n",
      "avg / total       0.54      0.62      0.55        32\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:45:58.125192: Epoch   0 Batch  400/3125   test_loss = 0.523  accuracy = 0.75  auc = 0.6282051282051282\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.82      0.88      0.85        26\n",
      "        1.0       0.25      0.17      0.20         6\n",
      "\n",
      "avg / total       0.71      0.75      0.73        32\n",
      "\n",
      "2017-12-30T20:45:58.225354: Epoch   0 Batch  425/3125   test_loss = 0.483  accuracy = 0.84375  auc = 0.711111111111111\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.92      0.89      0.91        27\n",
      "        1.0       0.50      0.60      0.55         5\n",
      "\n",
      "avg / total       0.86      0.84      0.85        32\n",
      "\n",
      "2017-12-30T20:45:58.321660: Epoch   0 Batch  450/3125   test_loss = 0.435  accuracy = 0.875  auc = 0.6428571428571428\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.90      0.96      0.93        28\n",
      "        1.0       0.50      0.25      0.33         4\n",
      "\n",
      "avg / total       0.85      0.88      0.86        32\n",
      "\n",
      "2017-12-30T20:45:58.425192: Epoch   0 Batch  475/3125   test_loss = 0.482  accuracy = 0.78125  auc = 0.75\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.81      0.92      0.86        24\n",
      "        1.0       0.60      0.38      0.46         8\n",
      "\n",
      "avg / total       0.76      0.78      0.76        32\n",
      "\n",
      "2017-12-30T20:45:58.520813: Epoch   0 Batch  500/3125   test_loss = 0.508  accuracy = 0.71875  auc = 0.8181818181818181\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.73      0.90      0.81        21\n",
      "        1.0       0.67      0.36      0.47        11\n",
      "\n",
      "avg / total       0.71      0.72      0.69        32\n",
      "\n",
      "2017-12-30T20:45:58.617479: Epoch   0 Batch  525/3125   test_loss = 0.399  accuracy = 0.84375  auc = 0.8385416666666666\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.83      1.00      0.91        24\n",
      "        1.0       1.00      0.38      0.55         8\n",
      "\n",
      "avg / total       0.87      0.84      0.82        32\n",
      "\n",
      "2017-12-30T20:45:58.721498: Epoch   0 Batch  550/3125   test_loss = 0.420  accuracy = 0.78125  auc = 0.8685714285714285\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.82      0.92      0.87        25\n",
      "        1.0       0.50      0.29      0.36         7\n",
      "\n",
      "avg / total       0.75      0.78      0.76        32\n",
      "\n",
      "2017-12-30T20:45:58.819107: Epoch   0 Batch  575/3125   test_loss = 0.454  accuracy = 0.78125  auc = 0.8067632850241546\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.83      0.87      0.85        23\n",
      "        1.0       0.62      0.56      0.59         9\n",
      "\n",
      "avg / total       0.77      0.78      0.78        32\n",
      "\n",
      "2017-12-30T20:45:58.919968: Epoch   0 Batch  600/3125   test_loss = 0.397  accuracy = 0.78125  auc = 0.9130434782608696\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.81      0.91      0.86        23\n",
      "        1.0       0.67      0.44      0.53         9\n",
      "\n",
      "avg / total       0.77      0.78      0.77        32\n",
      "\n",
      "2017-12-30T20:45:59.026510: Epoch   0 Batch  625/3125   test_loss = 0.527  accuracy = 0.75  auc = 0.6514285714285714\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.77      0.96      0.86        25\n",
      "        1.0       0.00      0.00      0.00         7\n",
      "\n",
      "avg / total       0.60      0.75      0.67        32\n",
      "\n",
      "2017-12-30T20:45:59.168186: Epoch   0 Batch  650/3125   test_loss = 0.391  accuracy = 0.8125  auc = 0.8333333333333333\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.92      0.85      0.88        26\n",
      "        1.0       0.50      0.67      0.57         6\n",
      "\n",
      "avg / total       0.84      0.81      0.82        32\n",
      "\n",
      "2017-12-30T20:45:59.272944: Epoch   0 Batch  675/3125   test_loss = 0.369  accuracy = 0.8125  auc = 0.7931034482758621\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.93      0.86      0.89        29\n",
      "        1.0       0.20      0.33      0.25         3\n",
      "\n",
      "avg / total       0.86      0.81      0.83        32\n",
      "\n",
      "2017-12-30T20:45:59.369430: Epoch   0 Batch  700/3125   test_loss = 0.430  accuracy = 0.8125  auc = 0.6987179487179487\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.92      0.89        26\n",
      "        1.0       0.50      0.33      0.40         6\n",
      "\n",
      "avg / total       0.79      0.81      0.80        32\n",
      "\n",
      "2017-12-30T20:45:59.466844: Epoch   0 Batch  725/3125   test_loss = 0.536  accuracy = 0.625  auc = 0.7239583333333334\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.77      0.71      0.74        24\n",
      "        1.0       0.30      0.38      0.33         8\n",
      "\n",
      "avg / total       0.65      0.62      0.64        32\n",
      "\n",
      "2017-12-30T20:45:59.571573: Epoch   0 Batch  750/3125   test_loss = 0.483  accuracy = 0.71875  auc = 0.72\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.81      0.84      0.82        25\n",
      "        1.0       0.33      0.29      0.31         7\n",
      "\n",
      "avg / total       0.70      0.72      0.71        32\n",
      "\n",
      "2017-12-30T20:45:59.669111: Epoch   0 Batch  775/3125   test_loss = 0.330  accuracy = 0.78125  auc = 0.9230769230769231\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.83      0.92      0.87        26\n",
      "        1.0       0.33      0.17      0.22         6\n",
      "\n",
      "avg / total       0.73      0.78      0.75        32\n",
      "\n",
      "2017-12-30T20:45:59.766711: Epoch   0 Batch  800/3125   test_loss = 0.404  accuracy = 0.875  auc = 0.6666666666666665\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.93      0.93      0.93        29\n",
      "        1.0       0.33      0.33      0.33         3\n",
      "\n",
      "avg / total       0.88      0.88      0.88        32\n",
      "\n",
      "2017-12-30T20:45:59.872467: Epoch   0 Batch  825/3125   test_loss = 0.450  accuracy = 0.78125  auc = 0.796875\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.84      0.88      0.86        24\n",
      "        1.0       0.57      0.50      0.53         8\n",
      "\n",
      "avg / total       0.77      0.78      0.78        32\n",
      "\n",
      "2017-12-30T20:45:59.972185: Epoch   0 Batch  850/3125   test_loss = 0.521  accuracy = 0.78125  auc = 0.7485714285714286\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.85      0.88      0.86        25\n",
      "        1.0       0.50      0.43      0.46         7\n",
      "\n",
      "avg / total       0.77      0.78      0.77        32\n",
      "\n",
      "2017-12-30T20:46:00.116733: Epoch   0 Batch  875/3125   test_loss = 0.344  accuracy = 0.875  auc = 0.8589743589743589\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.87      1.00      0.93        26\n",
      "        1.0       1.00      0.33      0.50         6\n",
      "\n",
      "avg / total       0.89      0.88      0.85        32\n",
      "\n",
      "2017-12-30T20:46:00.217339: Epoch   0 Batch  900/3125   test_loss = 0.478  accuracy = 0.71875  auc = 0.8115942028985507\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.75      0.91      0.82        23\n",
      "        1.0       0.50      0.22      0.31         9\n",
      "\n",
      "avg / total       0.68      0.72      0.68        32\n",
      "\n",
      "2017-12-30T20:46:00.312219: Epoch   0 Batch  925/3125   test_loss = 0.327  accuracy = 0.84375  auc = 0.8974358974358974\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.96      0.91        26\n",
      "        1.0       0.67      0.33      0.44         6\n",
      "\n",
      "avg / total       0.83      0.84      0.82        32\n",
      "\n",
      "2017-12-30T20:46:00.415304: Epoch   0 Batch  950/3125   test_loss = 0.510  accuracy = 0.78125  auc = 0.7922077922077922\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.79      0.90      0.84        21\n",
      "        1.0       0.75      0.55      0.63        11\n",
      "\n",
      "avg / total       0.78      0.78      0.77        32\n",
      "\n",
      "2017-12-30T20:46:00.512442: Epoch   0 Batch  975/3125   test_loss = 0.295  accuracy = 0.875  auc = 0.874074074074074\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.93      0.93      0.93        27\n",
      "        1.0       0.60      0.60      0.60         5\n",
      "\n",
      "avg / total       0.88      0.88      0.88        32\n",
      "\n",
      "2017-12-30T20:46:00.609767: Epoch   0 Batch 1000/3125   test_loss = 0.409  accuracy = 0.8125  auc = 0.8743961352657005\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.81      0.96      0.88        23\n",
      "        1.0       0.80      0.44      0.57         9\n",
      "\n",
      "avg / total       0.81      0.81      0.79        32\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:46:00.712791: Epoch   0 Batch 1025/3125   test_loss = 0.473  accuracy = 0.75  auc = 0.9041666666666666\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.73      0.95      0.83        20\n",
      "        1.0       0.83      0.42      0.56        12\n",
      "\n",
      "avg / total       0.77      0.75      0.72        32\n",
      "\n",
      "2017-12-30T20:46:00.808243: Epoch   0 Batch 1050/3125   test_loss = 0.390  accuracy = 0.84375  auc = 0.9166666666666667\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.83      1.00      0.91        24\n",
      "        1.0       1.00      0.38      0.55         8\n",
      "\n",
      "avg / total       0.87      0.84      0.82        32\n",
      "\n",
      "2017-12-30T20:46:00.909151: Epoch   0 Batch 1075/3125   test_loss = 0.443  accuracy = 0.8125  auc = 0.7307692307692308\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.92      0.89        26\n",
      "        1.0       0.50      0.33      0.40         6\n",
      "\n",
      "avg / total       0.79      0.81      0.80        32\n",
      "\n",
      "2017-12-30T20:46:01.013571: Epoch   0 Batch 1100/3125   test_loss = 0.403  accuracy = 0.78125  auc = 0.8599033816425121\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.79      0.96      0.86        23\n",
      "        1.0       0.75      0.33      0.46         9\n",
      "\n",
      "avg / total       0.78      0.78      0.75        32\n",
      "\n",
      "2017-12-30T20:46:01.158855: Epoch   0 Batch 1125/3125   test_loss = 0.386  accuracy = 0.84375  auc = 0.8985507246376812\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.82      1.00      0.90        23\n",
      "        1.0       1.00      0.44      0.62         9\n",
      "\n",
      "avg / total       0.87      0.84      0.82        32\n",
      "\n",
      "2017-12-30T20:46:01.262513: Epoch   0 Batch 1150/3125   test_loss = 0.522  accuracy = 0.84375  auc = 0.7101449275362319\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.82      1.00      0.90        23\n",
      "        1.0       1.00      0.44      0.62         9\n",
      "\n",
      "avg / total       0.87      0.84      0.82        32\n",
      "\n",
      "2017-12-30T20:46:01.359843: Epoch   0 Batch 1175/3125   test_loss = 0.571  accuracy = 0.6875  auc = 0.6954545454545454\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.71      0.91      0.80        22\n",
      "        1.0       0.50      0.20      0.29        10\n",
      "\n",
      "avg / total       0.65      0.69      0.64        32\n",
      "\n",
      "2017-12-30T20:46:01.455416: Epoch   0 Batch 1200/3125   test_loss = 0.598  accuracy = 0.71875  auc = 0.7056277056277056\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.70      1.00      0.82        21\n",
      "        1.0       1.00      0.18      0.31        11\n",
      "\n",
      "avg / total       0.80      0.72      0.65        32\n",
      "\n",
      "2017-12-30T20:46:01.560314: Epoch   0 Batch 1225/3125   test_loss = 0.435  accuracy = 0.78125  auc = 0.796875\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.79      0.96      0.87        24\n",
      "        1.0       0.67      0.25      0.36         8\n",
      "\n",
      "avg / total       0.76      0.78      0.74        32\n",
      "\n",
      "2017-12-30T20:46:01.658310: Epoch   0 Batch 1250/3125   test_loss = 0.537  accuracy = 0.6875  auc = 0.671875\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.73      0.92      0.81        24\n",
      "        1.0       0.00      0.00      0.00         8\n",
      "\n",
      "avg / total       0.55      0.69      0.61        32\n",
      "\n",
      "2017-12-30T20:46:01.754539: Epoch   0 Batch 1275/3125   test_loss = 0.455  accuracy = 0.78125  auc = 0.8115942028985508\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.81      0.91      0.86        23\n",
      "        1.0       0.67      0.44      0.53         9\n",
      "\n",
      "avg / total       0.77      0.78      0.77        32\n",
      "\n",
      "2017-12-30T20:46:01.860975: Epoch   0 Batch 1300/3125   test_loss = 0.539  accuracy = 0.65625  auc = 0.748917748917749\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.71      0.81      0.76        21\n",
      "        1.0       0.50      0.36      0.42        11\n",
      "\n",
      "avg / total       0.64      0.66      0.64        32\n",
      "\n",
      "2017-12-30T20:46:01.962907: Epoch   0 Batch 1325/3125   test_loss = 0.500  accuracy = 0.84375  auc = 0.4464285714285714\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.90      0.93      0.91        28\n",
      "        1.0       0.33      0.25      0.29         4\n",
      "\n",
      "avg / total       0.83      0.84      0.83        32\n",
      "\n",
      "2017-12-30T20:46:02.073523: Epoch   0 Batch 1350/3125   test_loss = 0.518  accuracy = 0.75  auc = 0.8125\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.71      1.00      0.83        20\n",
      "        1.0       1.00      0.33      0.50        12\n",
      "\n",
      "avg / total       0.82      0.75      0.71        32\n",
      "\n",
      "2017-12-30T20:46:02.202697: Epoch   0 Batch 1375/3125   test_loss = 0.531  accuracy = 0.6875  auc = 0.7777777777777778\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.72      0.91      0.81        23\n",
      "        1.0       0.33      0.11      0.17         9\n",
      "\n",
      "avg / total       0.61      0.69      0.63        32\n",
      "\n",
      "2017-12-30T20:46:02.302359: Epoch   0 Batch 1400/3125   test_loss = 0.310  accuracy = 0.90625  auc = 0.9285714285714286\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.96      0.93      0.95        28\n",
      "        1.0       0.60      0.75      0.67         4\n",
      "\n",
      "avg / total       0.92      0.91      0.91        32\n",
      "\n",
      "2017-12-30T20:46:02.400834: Epoch   0 Batch 1425/3125   test_loss = 0.548  accuracy = 0.71875  auc = 0.5833333333333334\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.81      0.85      0.83        26\n",
      "        1.0       0.20      0.17      0.18         6\n",
      "\n",
      "avg / total       0.70      0.72      0.71        32\n",
      "\n",
      "2017-12-30T20:46:02.496101: Epoch   0 Batch 1450/3125   test_loss = 0.392  accuracy = 0.78125  auc = 0.8854166666666666\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.79      0.96      0.87        24\n",
      "        1.0       0.67      0.25      0.36         8\n",
      "\n",
      "avg / total       0.76      0.78      0.74        32\n",
      "\n",
      "2017-12-30T20:46:02.598380: Epoch   0 Batch 1475/3125   test_loss = 0.366  accuracy = 0.875  auc = 0.7333333333333334\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.87      1.00      0.93        27\n",
      "        1.0       1.00      0.20      0.33         5\n",
      "\n",
      "avg / total       0.89      0.88      0.84        32\n",
      "\n",
      "2017-12-30T20:46:02.692972: Epoch   0 Batch 1500/3125   test_loss = 0.326  accuracy = 0.8125  auc = 0.8782051282051282\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.92      0.89        26\n",
      "        1.0       0.50      0.33      0.40         6\n",
      "\n",
      "avg / total       0.79      0.81      0.80        32\n",
      "\n",
      "2017-12-30T20:46:02.789402: Epoch   0 Batch 1525/3125   test_loss = 0.384  accuracy = 0.84375  auc = 0.7948717948717949\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.96      0.91        26\n",
      "        1.0       0.67      0.33      0.44         6\n",
      "\n",
      "avg / total       0.83      0.84      0.82        32\n",
      "\n",
      "2017-12-30T20:46:02.896679: Epoch   0 Batch 1550/3125   test_loss = 0.395  accuracy = 0.8125  auc = 0.8645833333333335\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.82      0.96      0.88        24\n",
      "        1.0       0.75      0.38      0.50         8\n",
      "\n",
      "avg / total       0.80      0.81      0.79        32\n",
      "\n",
      "2017-12-30T20:46:02.996215: Epoch   0 Batch 1575/3125   test_loss = 0.598  accuracy = 0.71875  auc = 0.7408906882591093\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.69      0.95      0.80        19\n",
      "        1.0       0.83      0.38      0.53        13\n",
      "\n",
      "avg / total       0.75      0.72      0.69        32\n",
      "\n",
      "2017-12-30T20:46:03.130904: Epoch   0 Batch 1600/3125   test_loss = 0.432  accuracy = 0.8125  auc = 0.8057142857142857\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.85      0.92      0.88        25\n",
      "        1.0       0.60      0.43      0.50         7\n",
      "\n",
      "avg / total       0.80      0.81      0.80        32\n",
      "\n",
      "2017-12-30T20:46:03.238779: Epoch   0 Batch 1625/3125   test_loss = 0.659  accuracy = 0.71875  auc = 0.7044534412955465\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.69      0.95      0.80        19\n",
      "        1.0       0.83      0.38      0.53        13\n",
      "\n",
      "avg / total       0.75      0.72      0.69        32\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:46:03.335108: Epoch   0 Batch 1650/3125   test_loss = 0.545  accuracy = 0.71875  auc = 0.7246376811594203\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.77      0.87      0.82        23\n",
      "        1.0       0.50      0.33      0.40         9\n",
      "\n",
      "avg / total       0.69      0.72      0.70        32\n",
      "\n",
      "2017-12-30T20:46:03.438614: Epoch   0 Batch 1675/3125   test_loss = 0.477  accuracy = 0.75  auc = 0.7447916666666666\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.79      0.92      0.85        24\n",
      "        1.0       0.50      0.25      0.33         8\n",
      "\n",
      "avg / total       0.71      0.75      0.72        32\n",
      "\n",
      "2017-12-30T20:46:03.534411: Epoch   0 Batch 1700/3125   test_loss = 0.328  accuracy = 0.875  auc = 0.9363636363636363\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.85      1.00      0.92        22\n",
      "        1.0       1.00      0.60      0.75        10\n",
      "\n",
      "avg / total       0.89      0.88      0.86        32\n",
      "\n",
      "2017-12-30T20:46:03.638078: Epoch   0 Batch 1725/3125   test_loss = 0.371  accuracy = 0.84375  auc = 0.6781609195402298\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.93      0.90      0.91        29\n",
      "        1.0       0.25      0.33      0.29         3\n",
      "\n",
      "avg / total       0.86      0.84      0.85        32\n",
      "\n",
      "2017-12-30T20:46:03.733826: Epoch   0 Batch 1750/3125   test_loss = 0.330  accuracy = 0.84375  auc = 0.9230769230769231\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.92      0.88      0.90        26\n",
      "        1.0       0.57      0.67      0.62         6\n",
      "\n",
      "avg / total       0.85      0.84      0.85        32\n",
      "\n",
      "2017-12-30T20:46:03.836390: Epoch   0 Batch 1775/3125   test_loss = 0.384  accuracy = 0.75  auc = 0.8593750000000001\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.81      0.88      0.84        24\n",
      "        1.0       0.50      0.38      0.43         8\n",
      "\n",
      "avg / total       0.73      0.75      0.74        32\n",
      "\n",
      "2017-12-30T20:46:03.936860: Epoch   0 Batch 1800/3125   test_loss = 0.244  accuracy = 0.9375  auc = 0.9851851851851853\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.96      0.96      0.96        27\n",
      "        1.0       0.80      0.80      0.80         5\n",
      "\n",
      "avg / total       0.94      0.94      0.94        32\n",
      "\n",
      "2017-12-30T20:46:04.039015: Epoch   0 Batch 1825/3125   test_loss = 0.621  accuracy = 0.6875  auc = 0.8174603174603174\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.64      1.00      0.78        18\n",
      "        1.0       1.00      0.29      0.44        14\n",
      "\n",
      "avg / total       0.80      0.69      0.63        32\n",
      "\n",
      "2017-12-30T20:46:04.172138: Epoch   0 Batch 1850/3125   test_loss = 0.502  accuracy = 0.71875  auc = 0.7632850241545893\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.73      0.96      0.83        23\n",
      "        1.0       0.50      0.11      0.18         9\n",
      "\n",
      "avg / total       0.67      0.72      0.65        32\n",
      "\n",
      "2017-12-30T20:46:04.270906: Epoch   0 Batch 1875/3125   test_loss = 0.555  accuracy = 0.78125  auc = 0.6590909090909092\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.78      0.95      0.86        22\n",
      "        1.0       0.80      0.40      0.53        10\n",
      "\n",
      "avg / total       0.78      0.78      0.76        32\n",
      "\n",
      "2017-12-30T20:46:04.369035: Epoch   0 Batch 1900/3125   test_loss = 0.532  accuracy = 0.6875  auc = 0.7447916666666666\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.75      0.88      0.81        24\n",
      "        1.0       0.25      0.12      0.17         8\n",
      "\n",
      "avg / total       0.62      0.69      0.65        32\n",
      "\n",
      "2017-12-30T20:46:04.465269: Epoch   0 Batch 1925/3125   test_loss = 0.483  accuracy = 0.71875  auc = 0.8599033816425121\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.77      0.87      0.82        23\n",
      "        1.0       0.50      0.33      0.40         9\n",
      "\n",
      "avg / total       0.69      0.72      0.70        32\n",
      "\n",
      "2017-12-30T20:46:04.568348: Epoch   0 Batch 1950/3125   test_loss = 0.410  accuracy = 0.8125  auc = 0.8695652173913043\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.79      1.00      0.88        23\n",
      "        1.0       1.00      0.33      0.50         9\n",
      "\n",
      "avg / total       0.85      0.81      0.78        32\n",
      "\n",
      "2017-12-30T20:46:04.664196: Epoch   0 Batch 1975/3125   test_loss = 0.384  accuracy = 0.84375  auc = 0.8333333333333333\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.92      0.88      0.90        26\n",
      "        1.0       0.57      0.67      0.62         6\n",
      "\n",
      "avg / total       0.85      0.84      0.85        32\n",
      "\n",
      "2017-12-30T20:46:04.760282: Epoch   0 Batch 2000/3125   test_loss = 0.357  accuracy = 0.84375  auc = 0.8974358974358974\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.96      0.91        26\n",
      "        1.0       0.67      0.33      0.44         6\n",
      "\n",
      "avg / total       0.83      0.84      0.82        32\n",
      "\n",
      "2017-12-30T20:46:04.867516: Epoch   0 Batch 2025/3125   test_loss = 0.541  accuracy = 0.78125  auc = 0.5128205128205128\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.81      0.96      0.88        26\n",
      "        1.0       0.00      0.00      0.00         6\n",
      "\n",
      "avg / total       0.66      0.78      0.71        32\n",
      "\n",
      "2017-12-30T20:46:04.969303: Epoch   0 Batch 2050/3125   test_loss = 0.410  accuracy = 0.8125  auc = 0.8141025641025641\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.92      0.89        26\n",
      "        1.0       0.50      0.33      0.40         6\n",
      "\n",
      "avg / total       0.79      0.81      0.80        32\n",
      "\n",
      "2017-12-30T20:46:05.084011: Epoch   0 Batch 2075/3125   test_loss = 0.419  accuracy = 0.8125  auc = 0.8454106280193237\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.84      0.91      0.87        23\n",
      "        1.0       0.71      0.56      0.63         9\n",
      "\n",
      "avg / total       0.80      0.81      0.80        32\n",
      "\n",
      "2017-12-30T20:46:05.211418: Epoch   0 Batch 2100/3125   test_loss = 0.435  accuracy = 0.75  auc = 0.8695652173913044\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.78      0.91      0.84        23\n",
      "        1.0       0.60      0.33      0.43         9\n",
      "\n",
      "avg / total       0.73      0.75      0.72        32\n",
      "\n",
      "2017-12-30T20:46:05.311176: Epoch   0 Batch 2125/3125   test_loss = 0.523  accuracy = 0.78125  auc = 0.6923076923076924\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.85      0.88      0.87        26\n",
      "        1.0       0.40      0.33      0.36         6\n",
      "\n",
      "avg / total       0.77      0.78      0.77        32\n",
      "\n",
      "2017-12-30T20:46:05.410442: Epoch   0 Batch 2150/3125   test_loss = 0.411  accuracy = 0.84375  auc = 0.7756410256410258\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.96      0.91        26\n",
      "        1.0       0.67      0.33      0.44         6\n",
      "\n",
      "avg / total       0.83      0.84      0.82        32\n",
      "\n",
      "2017-12-30T20:46:05.507550: Epoch   0 Batch 2175/3125   test_loss = 0.323  accuracy = 0.875  auc = 0.9314285714285714\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.92      0.92      0.92        25\n",
      "        1.0       0.71      0.71      0.71         7\n",
      "\n",
      "avg / total       0.88      0.88      0.88        32\n",
      "\n",
      "2017-12-30T20:46:05.611483: Epoch   0 Batch 2200/3125   test_loss = 0.303  accuracy = 0.90625  auc = 0.967948717948718\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.93      0.96      0.94        26\n",
      "        1.0       0.80      0.67      0.73         6\n",
      "\n",
      "avg / total       0.90      0.91      0.90        32\n",
      "\n",
      "2017-12-30T20:46:05.707313: Epoch   0 Batch 2225/3125   test_loss = 0.522  accuracy = 0.6875  auc = 0.7727272727272727\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.73      0.86      0.79        22\n",
      "        1.0       0.50      0.30      0.37        10\n",
      "\n",
      "avg / total       0.66      0.69      0.66        32\n",
      "\n",
      "2017-12-30T20:46:05.803906: Epoch   0 Batch 2250/3125   test_loss = 0.517  accuracy = 0.78125  auc = 0.8083333333333333\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.74      1.00      0.85        20\n",
      "        1.0       1.00      0.42      0.59        12\n",
      "\n",
      "avg / total       0.84      0.78      0.75        32\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:46:05.915735: Epoch   0 Batch 2275/3125   test_loss = 0.420  accuracy = 0.875  auc = 0.9230769230769231\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.83      1.00      0.90        19\n",
      "        1.0       1.00      0.69      0.82        13\n",
      "\n",
      "avg / total       0.90      0.88      0.87        32\n",
      "\n",
      "2017-12-30T20:46:06.012810: Epoch   0 Batch 2300/3125   test_loss = 0.561  accuracy = 0.75  auc = 0.7402597402597403\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.74      0.95      0.83        21\n",
      "        1.0       0.80      0.36      0.50        11\n",
      "\n",
      "avg / total       0.76      0.75      0.72        32\n",
      "\n",
      "2017-12-30T20:46:06.160072: Epoch   0 Batch 2325/3125   test_loss = 0.449  accuracy = 0.78125  auc = 0.8125\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.79      0.96      0.87        24\n",
      "        1.0       0.67      0.25      0.36         8\n",
      "\n",
      "avg / total       0.76      0.78      0.74        32\n",
      "\n",
      "2017-12-30T20:46:06.258813: Epoch   0 Batch 2350/3125   test_loss = 0.376  accuracy = 0.84375  auc = 0.8628571428571428\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.83      1.00      0.91        25\n",
      "        1.0       1.00      0.29      0.44         7\n",
      "\n",
      "avg / total       0.87      0.84      0.81        32\n",
      "\n",
      "2017-12-30T20:46:06.353488: Epoch   0 Batch 2375/3125   test_loss = 0.415  accuracy = 0.875  auc = 0.828125\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      1.00      0.92        24\n",
      "        1.0       1.00      0.50      0.67         8\n",
      "\n",
      "avg / total       0.89      0.88      0.86        32\n",
      "\n",
      "2017-12-30T20:46:06.456927: Epoch   0 Batch 2400/3125   test_loss = 0.296  accuracy = 0.875  auc = 0.9407407407407408\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.90      0.96      0.93        27\n",
      "        1.0       0.67      0.40      0.50         5\n",
      "\n",
      "avg / total       0.86      0.88      0.86        32\n",
      "\n",
      "2017-12-30T20:46:06.551657: Epoch   0 Batch 2425/3125   test_loss = 0.470  accuracy = 0.78125  auc = 0.7771428571428571\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.82      0.92      0.87        25\n",
      "        1.0       0.50      0.29      0.36         7\n",
      "\n",
      "avg / total       0.75      0.78      0.76        32\n",
      "\n",
      "2017-12-30T20:46:06.646638: Epoch   0 Batch 2450/3125   test_loss = 0.565  accuracy = 0.75  auc = 0.6956521739130435\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.76      0.96      0.85        23\n",
      "        1.0       0.67      0.22      0.33         9\n",
      "\n",
      "avg / total       0.73      0.75      0.70        32\n",
      "\n",
      "2017-12-30T20:46:06.750003: Epoch   0 Batch 2475/3125   test_loss = 0.561  accuracy = 0.6875  auc = 0.7101449275362319\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.72      0.91      0.81        23\n",
      "        1.0       0.33      0.11      0.17         9\n",
      "\n",
      "avg / total       0.61      0.69      0.63        32\n",
      "\n",
      "2017-12-30T20:46:06.848641: Epoch   0 Batch 2500/3125   test_loss = 0.519  accuracy = 0.78125  auc = 0.734375\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.77      1.00      0.87        24\n",
      "        1.0       1.00      0.12      0.22         8\n",
      "\n",
      "avg / total       0.83      0.78      0.71        32\n",
      "\n",
      "2017-12-30T20:46:06.948869: Epoch   0 Batch 2525/3125   test_loss = 0.675  accuracy = 0.6875  auc = 0.7380952380952381\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.67      0.89      0.76        18\n",
      "        1.0       0.75      0.43      0.55        14\n",
      "\n",
      "avg / total       0.70      0.69      0.67        32\n",
      "\n",
      "2017-12-30T20:46:07.060402: Epoch   0 Batch 2550/3125   test_loss = 0.323  accuracy = 0.875  auc = 0.9310344827586207\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.93      0.93      0.93        29\n",
      "        1.0       0.33      0.33      0.33         3\n",
      "\n",
      "avg / total       0.88      0.88      0.88        32\n",
      "\n",
      "2017-12-30T20:46:07.197052: Epoch   0 Batch 2575/3125   test_loss = 0.433  accuracy = 0.78125  auc = 0.7942857142857143\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.82      0.92      0.87        25\n",
      "        1.0       0.50      0.29      0.36         7\n",
      "\n",
      "avg / total       0.75      0.78      0.76        32\n",
      "\n",
      "2017-12-30T20:46:07.300396: Epoch   0 Batch 2600/3125   test_loss = 0.327  accuracy = 0.875  auc = 0.9107142857142857\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.96      0.89      0.93        28\n",
      "        1.0       0.50      0.75      0.60         4\n",
      "\n",
      "avg / total       0.90      0.88      0.89        32\n",
      "\n",
      "2017-12-30T20:46:07.396936: Epoch   0 Batch 2625/3125   test_loss = 0.508  accuracy = 0.71875  auc = 0.8067632850241546\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.75      0.91      0.82        23\n",
      "        1.0       0.50      0.22      0.31         9\n",
      "\n",
      "avg / total       0.68      0.72      0.68        32\n",
      "\n",
      "2017-12-30T20:46:07.493027: Epoch   0 Batch 2650/3125   test_loss = 0.492  accuracy = 0.75  auc = 0.7971014492753623\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.76      0.96      0.85        23\n",
      "        1.0       0.67      0.22      0.33         9\n",
      "\n",
      "avg / total       0.73      0.75      0.70        32\n",
      "\n",
      "2017-12-30T20:46:07.596522: Epoch   0 Batch 2675/3125   test_loss = 0.702  accuracy = 0.59375  auc = 0.5603864734299517\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.68      0.83      0.75        23\n",
      "        1.0       0.00      0.00      0.00         9\n",
      "\n",
      "avg / total       0.49      0.59      0.54        32\n",
      "\n",
      "2017-12-30T20:46:07.694581: Epoch   0 Batch 2700/3125   test_loss = 0.464  accuracy = 0.8125  auc = 0.7239583333333334\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.80      1.00      0.89        24\n",
      "        1.0       1.00      0.25      0.40         8\n",
      "\n",
      "avg / total       0.85      0.81      0.77        32\n",
      "\n",
      "2017-12-30T20:46:07.791573: Epoch   0 Batch 2725/3125   test_loss = 0.453  accuracy = 0.78125  auc = 0.7864583333333333\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.77      1.00      0.87        24\n",
      "        1.0       1.00      0.12      0.22         8\n",
      "\n",
      "avg / total       0.83      0.78      0.71        32\n",
      "\n",
      "2017-12-30T20:46:07.899617: Epoch   0 Batch 2750/3125   test_loss = 0.356  accuracy = 0.84375  auc = 0.8125\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.90      0.93      0.91        28\n",
      "        1.0       0.33      0.25      0.29         4\n",
      "\n",
      "avg / total       0.83      0.84      0.83        32\n",
      "\n",
      "2017-12-30T20:46:07.999013: Epoch   0 Batch 2775/3125   test_loss = 0.653  accuracy = 0.65625  auc = 0.7206477732793523\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.65      0.89      0.76        19\n",
      "        1.0       0.67      0.31      0.42        13\n",
      "\n",
      "avg / total       0.66      0.66      0.62        32\n",
      "\n",
      "2017-12-30T20:46:08.138036: Epoch   0 Batch 2800/3125   test_loss = 0.540  accuracy = 0.6875  auc = 0.8663967611336033\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.66      1.00      0.79        19\n",
      "        1.0       1.00      0.23      0.38        13\n",
      "\n",
      "avg / total       0.80      0.69      0.62        32\n",
      "\n",
      "2017-12-30T20:46:08.239946: Epoch   0 Batch 2825/3125   test_loss = 0.384  accuracy = 0.8125  auc = 0.8397435897435898\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.86      0.92      0.89        26\n",
      "        1.0       0.50      0.33      0.40         6\n",
      "\n",
      "avg / total       0.79      0.81      0.80        32\n",
      "\n",
      "2017-12-30T20:46:08.335412: Epoch   0 Batch 2850/3125   test_loss = 0.499  accuracy = 0.71875  auc = 0.8019323671497585\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.75      0.91      0.82        23\n",
      "        1.0       0.50      0.22      0.31         9\n",
      "\n",
      "avg / total       0.68      0.72      0.68        32\n",
      "\n",
      "2017-12-30T20:46:08.438502: Epoch   0 Batch 2875/3125   test_loss = 0.531  accuracy = 0.75  auc = 0.7409090909090909\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.75      0.95      0.84        22\n",
      "        1.0       0.75      0.30      0.43        10\n",
      "\n",
      "avg / total       0.75      0.75      0.71        32\n",
      "\n",
      "2017-12-30T20:46:08.535287: Epoch   0 Batch 2900/3125   test_loss = 0.317  accuracy = 0.84375  auc = 0.9257142857142857\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.83      1.00      0.91        25\n",
      "        1.0       1.00      0.29      0.44         7\n",
      "\n",
      "avg / total       0.87      0.84      0.81        32\n",
      "\n",
      "2017-12-30T20:46:08.631813: Epoch   0 Batch 2925/3125   test_loss = 0.457  accuracy = 0.8125  auc = 0.7239583333333334\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.80      1.00      0.89        24\n",
      "        1.0       1.00      0.25      0.40         8\n",
      "\n",
      "avg / total       0.85      0.81      0.77        32\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2017-12-30T20:46:08.734997: Epoch   0 Batch 2950/3125   test_loss = 0.352  accuracy = 0.84375  auc = 0.9010416666666667\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.85      0.96      0.90        24\n",
      "        1.0       0.80      0.50      0.62         8\n",
      "\n",
      "avg / total       0.84      0.84      0.83        32\n",
      "\n",
      "2017-12-30T20:46:08.831589: Epoch   0 Batch 2975/3125   test_loss = 0.483  accuracy = 0.78125  auc = 0.8441558441558441\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.79      0.90      0.84        21\n",
      "        1.0       0.75      0.55      0.63        11\n",
      "\n",
      "avg / total       0.78      0.78      0.77        32\n",
      "\n",
      "2017-12-30T20:46:08.934328: Epoch   0 Batch 3000/3125   test_loss = 0.437  accuracy = 0.8125  auc = 0.7371428571428571\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.83      0.96      0.89        25\n",
      "        1.0       0.67      0.29      0.40         7\n",
      "\n",
      "avg / total       0.79      0.81      0.78        32\n",
      "\n",
      "2017-12-30T20:46:09.043729: Epoch   0 Batch 3025/3125   test_loss = 0.534  accuracy = 0.71875  auc = 0.7590909090909091\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.72      0.95      0.82        22\n",
      "        1.0       0.67      0.20      0.31        10\n",
      "\n",
      "avg / total       0.71      0.72      0.66        32\n",
      "\n",
      "2017-12-30T20:46:09.182529: Epoch   0 Batch 3050/3125   test_loss = 0.500  accuracy = 0.78125  auc = 0.821256038647343\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.77      1.00      0.87        23\n",
      "        1.0       1.00      0.22      0.36         9\n",
      "\n",
      "avg / total       0.83      0.78      0.73        32\n",
      "\n",
      "2017-12-30T20:46:09.285597: Epoch   0 Batch 3075/3125   test_loss = 0.463  accuracy = 0.78125  auc = 0.8138528138528139\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.77      0.95      0.85        21\n",
      "        1.0       0.83      0.45      0.59        11\n",
      "\n",
      "avg / total       0.79      0.78      0.76        32\n",
      "\n",
      "2017-12-30T20:46:09.382021: Epoch   0 Batch 3100/3125   test_loss = 0.348  accuracy = 0.875  auc = 0.9111111111111111\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.93      0.93      0.93        27\n",
      "        1.0       0.60      0.60      0.60         5\n",
      "\n",
      "avg / total       0.88      0.88      0.88        32\n",
      "\n",
      "Model Trained and Saved\n"
     ]
    }
   ],
   "source": [
    "losses, acc_lst, pred_lst, load_dir = train_model(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "losses, acc_lst, pred_lst, load_dir = load_params()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 输出验证集上的训练信息\n",
    " - 平均准确率\n",
    " - 平均损失\n",
    " - 平均Auc\n",
    " - 预测的平均点击率\n",
    " - 精确率、召回率、F1 Score等信息\n",
    " \n",
    "因为数据中大部分都是负例，正例较少，如果模型全部猜0就能有75%的准确率，所以准确率这个指标是不可信的。\n",
    "\n",
    "我们需要关注正例的精确率和召回率，当然最主要还是要看LogLoss的值，因为比赛采用的评价指标是LogLoss，而不是采用AUC值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_info():\n",
    "    print(\"Test Mean Acc : {}\".format(np.mean(acc_lst['test'])))  #test_pred_mean\n",
    "    print(\"Test Mean Loss : {}\".format(np.mean(losses['test'])))  #test_pred_mean\n",
    "    print(\"Mean Auc : {}\".format(metrics.roc_auc_score(test_y[:-9], np.array(pred_lst).reshape(-1, 1))))\n",
    "    print(\"Mean prediction : {}\".format(np.mean(np.array(pred_lst).reshape(-1, 1))))\n",
    "    print(metrics.classification_report(test_y[:-9], np.float32(np.array(pred_lst).reshape(-1, 1) > 0.5)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Mean Acc : 0.7814300060272217\n",
      "Test Mean Loss : 0.46838584542274475\n",
      "Mean Auc : 0.7792937214782675\n",
      "Mean prediction : 0.2552148997783661\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        0.0       0.81      0.93      0.86     74426\n",
      "        1.0       0.63      0.34      0.45     25574\n",
      "\n",
      "avg / total       0.76      0.78      0.76    100000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "train_info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### TensorBoard中查看loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "tensorboard --logdir=/Users/chengstone/Downloads/cn-deep-learning-master/ctr/runs/1514636600\n",
    "<img src=\"tensorboard.png\"\\>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAusAAAH0CAYAAACEkWPuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3XecVNX9//HXobcFRVSsUYkoJvoz\nomKLsUSNRqMxajSJURPj16ixRKMxomAsWKISFXsEERsqAiJKXXpfelvq0pYFdoHtfc7vj9ldt8zs\nzM7cmXtn5v18PHjcZe6dM58tc+dzzz3nc4y1FhERERER8Z5WbgcgIiIiIiKBKVkXEREREfEoJesi\nIiIiIh6lZF1ERERExKOUrIuIiIiIeJSSdRERERERj1KyLiIiIiLiUUrWRUREREQ8Ssm6iIiIiIhH\nKVkXEREREfEoJesiIiIiIh6lZF1ERERExKOUrIuIiIiIeJSSdRERERERj1KyLiIiIiLiUUrWRURE\nREQ8qo3bAcSTMWYz0BXIcjkUEREREUluxwAF1tpjo2kkpZJ1oGvHjh279+nTp7vbgYiIiIhI8lqz\nZg2lpaVRt5NqyXpWnz59umdkZLgdh4iIiIgksb59+7J48eKsaNvRmHUREREREY9Ssi4iIiIi4lFK\n1kVEREREPErJuoiIiIiIRylZFxERERHxKCXrIiIiIiIepWRdRERERMSjUq3OuogkCJ/Px969eyks\nLKS8vBxrrdshibjGGEP79u1JS0uje/futGqlvjaRVKFkXUQ8x+fzsW3bNkpKStwORcQTrLWUlZVR\nVlZGcXExRx11lBJ2kRShZF1EPGfv3r2UlJTQpk0bevbsSefOnZWYSErz+XwUFxeTk5NDSUkJe/fu\npUePHm6HJSJxoE8/EfGcwsJCAHr27ElaWpoSdUl5rVq1Ii0tjZ49ewLfv0dEJPnpE1BEPKe8vByA\nzp07uxyJiLfUvidq3yMikvyUrIuI59ROJlWPukhDxhgATbgWSSH6JBQREUkQtcm6iKQOJesiIiIi\nIh6lZD3J6NaoiIiISPJQsp5E3py2kTOemcy7Mza5HYqIJImioiKMMVx55ZVRt3X66afTpUsXB6Jy\nzuuvv44xhi+++MLtUEREAlKyniSqfZbnv1tLblEFz4xf43Y4IhIlY0yL/g0bNsztkEVEJAa0KFKS\n8Gn4i0hSGTBgQJPHBg8eTH5+Pvfddx8HHHBAg32nnnpqTOLo3Lkza9ascaRH/Msvv1TJQRGRFlKy\nLiLiQQMHDmzy2LBhw8jPz+f+++/nmGOOiUscxhhOPPFER9r6wQ9+4Eg7IiKpRMNgRESSSO248NLS\nUvr3788Pf/hD2rVrxz333ANAXl4ezz33HD/72c84/PDDadeuHYceeii/+c1vWLx4cZP2go1Zf+ih\nhzDGsGjRIj766CP69u1Lx44d6dGjBzfffDO7d+8OGlt948aNwxjDf/7zHxYsWMBll11Gt27d6NKl\nCz//+c/JyMgI+H1u3bqVP/zhD/To0YNOnTrRt29fPvvsswbtRWvu3LlcffXV9OjRg/bt23Pcccdx\n//33s2fPnibHZmdnc99999G7d286derEgQceSJ8+ffjzn//Mtm3b6o7z+Xy8++679OvXjx49etCx\nY0eOPvporrjiCkaPHh11zCKSfNSzLiKSZHw+H1deeSWZmZlcdtllHHTQQXW92kuWLGHAgAFccMEF\nXH311XTr1o3NmzczduxYxo0bx6RJkzj//PPDfq0XXniBcePGcfXVV3PhhRcye/ZsRowYwcqVK1m0\naBGtW7cOq51Zs2bRv39/LrjgAv7yl7+wadMmRo8ezQUXXMDKlSsb9Mpv376ds88+m+zsbC6++GLO\nOOMMduzYwS233MLll1/esh9WECNHjuT3v/89rVu35vrrr+fII49k3rx5/Pe//2XMmDHMnj2bww8/\nHICCggL69etHdnY2l156Kddccw2VlZVs2bKFL774gptvvpmjjjoKgPvvv5/XXnuN448/nptuuoku\nXbqQnZ3N/PnzGT16NNdcc40j8YtI8lCyLiKSZEpLSyksLGTlypVNxrafdtpp5OTkcOCBBzZ4fOPG\njfTr148HH3yQhQsXhv1aU6ZMYenSpfTu3Rvwl4+95pprGDt2LBMmTOCKK64Iq50xY8bw+eefc911\n19U99tJLL/HQQw8xZMgQXnjhhbrHH3zwQbKzs/n3v//N448/Xvf4XXfdxXnnnRd27MHs3buX22+/\nHWMMs2bN4vTTT6/b9/jjj/P0009zzz33MGrUKAC++eYbtm/fTv/+/XnqqacatFVWVkZVVRXwfa96\nr169WLFiBe3bt29wbG5ubtSxi0jyUbIuIgnnmH9+43YIYct67peuvO6gQYOaJOoA3bt3D3h8r169\n+NWvfsXQoUPJy8vjoIMOCut1/vGPf9Ql6uAf43777bczduxYFixYEHayftlllzVI1AHuuOMOHnro\nIRYsWFD3WGFhIaNGjeKQQw7hH//4R4PjzzrrLK6//no+/fTTsF4zmM8//5zCwkL+8pe/NEjUAR57\n7DHee+89xowZQ25uLj169Kjb17FjxyZtdejQocH/jTG0a9cu4B2H+m2JiNTSmHURkSR05plnBt2X\nnp7Otddey5FHHkm7du3qyj8OHToU8I+/DlfjZBaoG/Kxb9++qNpJS0ujW7duDdpZuXIlVVVV9O3b\nt0kiDDjSs147dv+iiy5qsq9Dhw6cc845+Hw+li1bBsAll1zCwQcfzOOPP86VV17JkCFDWLp0KT6f\nr8FzW7VqxY033siaNWv48Y9/zOOPP87EiRMpLCyMOma3+XyWsspqt8MQSUrqWRcRSTKdOnUiLS0t\n4L4RI0bwxz/+kS5dunDJJZdw7LHH0rlzZ4wxTJw4kblz57aovGKg3vs2bfwfLdXV4Sdvgdqpbat+\nO/n5+QAceuihAY8P9nhL1L7GYYcdFnB/7eP79+8H/D3i8+fPZ+DAgYwbN45vvvmmLpZ7772XRx55\npK4n/e233+bEE0/kgw8+4Omnnwagbdu2/OpXv+Kll15KyIo5+aWVXP36LPYWV/D+rWdw+jGB797E\nUrXPMmPdHg4/oCMn9Az8ty+SqJSsi0jCcWtoSaIwxgTd179/f9LS0liyZAnHHXdcg33r169n7ty5\nsQ4vKl27dgVg165dAfcHe7wlunXrBkBOTk7A/Tt37mxwHMCxxx7LBx98gM/nY+XKlUyZMoXXX3+d\nxx57jNatW/PII48A/sT84Ycf5uGHHyYnJ4eZM2cyYsQIvvzyS9auXcuyZcvCnpTrFS9OWEtWXgkA\nN707j/XPhDf0yUkj5m1hwNhVGAMz/nEhR3XvFPcYRGJFw2BERFJEVVUVW7Zs4dRTT22SqFdWVno+\nUQc4+eSTadOmDRkZGZSVlTXZP2vWrKhf4yc/+QkA06ZNa7KvvLycuXPnYowJuBBVq1atOOWUU3jg\ngQcYN24cQNCSjD179uT6669nzJgxnHnmmaxatYoNGzZEHX+8ZeZ8P4ynstqdBfoGjF0FgLXwrFbx\nliSjZF1EJEW0adOGI444glWrVjWoPOLz+Xj00UfZvHmzi9GFJy0tjWuuuYbdu3fz4osvNtg3f/58\nPv/886hf44YbbqBLly4MHTq0blx6rUGDBrFz5866+usAS5cuZfv27U3aqe3l79TJ38tbVFTE9OnT\nmxxXXl5eN/Qm0CRVaRm3LhhEYkXDYEREUsgDDzzAQw89xCmnnMK1115Lq1atmD59OllZWVx++eV8\n++23bocY0ksvvcSsWbN44oknmDFjBmeccQbbt29n5MiRXHXVVYwePZpWrSLvi+revTvvvPMON998\nM2effTbXX389RxxxBPPmzSM9PZ2jjjqK119/ve74cePGMWDAAM477zxOOOEEevTowZYtWxgzZgyt\nW7fmoYceAvxj3C+44AJ69erFmWeeydFHH01JSQnfffcd69ev53e/+x1HH3101D8fEUkuStZFRFLI\n3//+d7p06cLrr7/O+++/T+fOnbngggsYOXIk7777bkIk60cffTTz5s3j0UcfZcKECcyaNYuTTjqJ\nDz74gNLSUkaPHl03tj1SN910E0cffTTPPfcc48aNo7CwkMMPP5y//e1v9O/fn0MOOaTu2F/96lfs\n2bOHmTNnMmrUKIqKijjssMO46qqrePDBB+sq3Rx00EE8++yzpKenM3PmTPbs2UPXrl05/vjjeeSR\nR7jllluiillEkpOxNnVuFxljMk477bTTgi1fncgqq30c/9j3H7KagCeJbM0a/5jTPn36uByJJJr7\n7ruPV199lVmzZnHuuee6HU5MeO39cf1bc1iY9X15TTc+f+qvvfDzPofy3i1NS4GKxFvfvn1ZvHjx\nYmtt32ja0Zh1ERFJOIFqwS9cuJB33nmHww8/nH79+rkQlYiI8zQMRkREEk6fPn047bTT+NGPfkSH\nDh3IzMysG8IzZMiQulrvIiKJTmczERFJOHfddRfjx4/no48+oqioiAMPPJArr7yShx9+mHPOOcft\n8EREHKNkXUREEs6gQYMYNGiQ22GIiMScY2PWjTFHGmPeN8ZkG2PKjTFZxpjBxpgDW9jOr40xU40x\n+40xZcaYNcaYJ4wxHZyKVUREREQkETiSrBtjegEZwG3AAuAVYBNwHzDXGHNQmO08BYwCzgBGA0OA\nAuBJYLIxRqtFiIiIiEjKcGoYzBvAIcC91trXah80xrwMPAA8A9zZXAPGmJ8AjwH7gb7W2k01jxvg\nVeAe4BFgoEMxi4iIJJRUKrcsIn5R96wbY44DLgWy8PeE1zcAKAZuNsZ0DtHUrwEDvFebqANY/5np\nX4AF/mqMaR1tzCLibf5rdPD5fC5HIuIttcl67XtERJKfE8NgLqrZTrTWNvhktdYWArOBTsBZIdrp\nWbPd1HhHTTu5+HvvT44qWhHxvPbt2wNQXFzsciQi3lL7nqh9j4hI8nNiGMwJNdt1Qfavx9/z3huY\n0kw7uTXbYxvvMMakAT1q/nsisLS5gIwxwZYoPbG554mIN6SlpVFWVkZOTg4AnTt3xhij3kRJSdZa\nrLUUFxfXvSfS0tJcjkpE4sWJZL1bzTY/yP7axw8I0c444FHgdmPMG9barHr7nsY/RAagRdVlRCTx\ndO/eneLiYkpKSti+fbvb4Yh4SqdOnejevbvbYYhInMSjznptkt3srBhr7RxjzNvA/wHLjTFfAnuB\nc/FXh1kF/AioDvWC1tq+AQPx97ifFn7oIuKGVq1acdRRR7F3714KCwspLy/XxDpJacYY2rdvT1pa\nGt27d6dVK8cqL4uIxzmRrNf2nHcLsr9ro+OCstbeaYxZANwB3FDzcAZwGfBn/Mn67shDTV4aHCDJ\nplWrVvTo0YMePXqEPlhERCRJOZGsZ9ZsewfZf3zNNtiY9gaste8D7zd+3BjzXs2XC1sUnYiIiKQQ\n3YWT5OLEfbT0mu2lxpgG7dVMDD0XKAXmRfoCxphLgR8A0621OyJtR0REREQkkUSdrFtrNwITgWOA\nuxvtfhLoDAy31tbVYDPGnGiMaVKZxRjTNcBjvYB38I9V/2e08YqIiIiIJAqnJpjeBcwBXjXGXAys\nAfoBF+If/vJYo+PX1GwbD7X+nzHmB/jHqe8DfghcBbQFbrfWRtw7LyIiIqlrX3EFB3RqqxKwknAc\nmU5e07t+OjAMf5L+INALeBU421qbF2ZT44BK/JNLHwLOAb4ETrPWDnMiVhEREUktgyev4ydPTeKO\nD4MtwyLiXY6VbrTWbgNuC/PYgJe11toPgA+ciklERERk8OT1AExavYvNucUc26OzyxGJhE+FWkVE\nRCRlFJdXuR2CSIsoWRcRERER8Sgl6yIiIiIiHqVkXURERETEo5Ssi4iIiIh4lJJ1ERERERGPUrIu\nIiIiIuJRStZFRERERDxKybqIiIiIiEcpWRcRERER8Sgl60nCuh2AiIiIiDhOybqIiIiIiEcpWRfP\nWJtTwCcLtlJQVul2KCIiIiKe0MbtAEQAisqr+PWQOZRWVrNk6z5euO7/uR2SiIgkIKtxoZJk1LMu\nnvD1smxKK6sBGLlou8vRiIiIJK6d+aXs2F/qdhjiECXrIiIiIklixfZ8zns+nZ8+P5WMLfvcDkcc\noGRdREREJEnc9XEG1T6Lz8LtHyx0OxxxgJJ1ERERkSSxK7+87ut9JSrYkAyUrIuIiLjI57Os2VmA\nz6eZkSLSlJL1JGHcDkBERCJy98eLufy/M7lzRIbboYiIBylZF5GUNW9THtcMmc3Lk9a5HYqksG9X\n5gAwcfUuqtW7LiKNqM66iKSsG9+ZB8DSbfu59KRD+fER3VyOSEREpCH1rIuIAJk5hW6HIB5UUeXj\n80XbmLgqB6vVdgLSj0UktpSsi4iIBPHx/C3844vl3PFhBvM27XU7HBFJQUrWRUTENbsLynh2/BrG\nLN3hdigBDfx6dd3XT4xZ6WIkIpKqNGZdRERc89AXy5mxbg8AvQ9No89hXV2OSJKdhu3Exr7iCrp1\nbEurVqpP5zT1rIuIiGtqE3WAscuyXYxERCI1eskOznhmMpcNnkFltc/tcJKOknURERFJGUYdv467\n/7OlVPks63cX8emCrW6Hk3SUrIuIiIiII3YXlrsdQtJRsi4iUVudXUBekU7QIqlIPdUisaVkXUSi\n8tnCrVzx6kzOfX6qEnYRcZ0uHiTZKFkXkag88uUKAMoqfbwyeZ3L0YiIiCQXJesi4pjCsiq3QxCR\nFFdbmrGsspqs3GJ3g3GBRbUpk43qrIuISMRyi8rJK6rghJ5pbociUqesspqfvZjOroJynrr6R26H\nIxIV9ayLJ2iIoUji2VVQxjnPTeWywTM8uwKppKYR87awq8A/h+bxMatcjkYkOkrWRUQkIk+NW01F\nlX8BlPs+XepyNCLf21dS4XYIIo5xLFk3xhxpjHnfGJNtjCk3xmQZYwYbYw5sYTvnGWPG1Dy/zBiz\n1Rgz3hjzC6diTRarsvMZkr6B7P2lbocikvBUQaLl8ksr3Q5BRCTpOTJm3RjTC5gDHAKMAdYCZwL3\nAb8wxpxrrc0Lo52/Am8AxcBXwHbgSOBa4HJjTH9r7TNOxJzoyiqr+dXrs6n2WSasymHUX89xOyQR\nkbCs31XI42NWcmyPLm6HIpJ0DAaScJJpZbWPr5bsoEPb1lx58mG0apU6PSxOTTB9A3+ifq+19rXa\nB40xLwMPAM8AdzbXgDGmLTAIKAP6Wmsz6+17FlgCPGaM+Y+1NuWLOa/NKaTa538zLt+e73I0ItGx\n1jJ2WTaFZVVc1/dIOrRt7XZIEkN/+mAh2/aWMm/TXrdDEZEE8dnCbfQfvRKAjm1bc8lJh7ocUfxE\nPQzGGHMccCmQBQxptHsA/l7ym40xnUM01R3oBqyrn6gDWGvXAOuAjoC6YpJQ8vUBSEtMW7eH+z5d\nSv/RKxk+N8vtcCTGtu2Nbuje9n0lvDltIxt2FzoUUWx8NH8LN/9vPvM2hbyxLHFk9YGTkGoTdYB/\nfbXCxUjiz4kx6xfVbCdaa331d1hrC4HZQCfgrBDt7Ab2AL2NMcfX32GM6Q0cDywNZziNiCSWp75e\nXff1s+PXuhiJBDNy0TZuemce6Zm73Q6FPw1byPPfreXXb8yhqtoX+gkuyMkv47GvVjJzfS43vjPP\n7XBEmlVV7WNvsSblepUTyfoJNdtgSxeur9n2bq4Ra60F7q6JKcMY84ExZpAxZjiQAawCrg8nIGNM\nRqB/wInhPF+kOZ8t3Mrjo1cm7MTewrJKTQyUFikoq+ThL5Yzd1Metw1d6HY4rNtVBPgX4dpd6M1R\nkZtyiyJ6nk3Sbt89heXM3pBLtc/y7Yqd/P2zpazckVhDOH0+y9S1u5izITepfk+lFdVc8J9pnPnM\nZMav2Ol2OBKAE2PWu9Vsg73rah8/IFRD1trPjTHZwCfAH+vt2gUMBTZFGqSIE5Zt288jX/pvv2Xm\nFDLyzrNdjqhlNuwu4urXZ+Gz8PmdZ/PjI7qFflIC2bC7kCVb93P5yYfRpX30p7c3pm1g7NJsHrik\nN5f9qKcDESamvCL1uEnkSiuqueSV6ewvqeTGM47i04XbABi3Yifrnr7c5ejC992qHO76aDEAn91x\nFv2OO8jliJzxzoxNbN/n73y666PFZD33S5cjksbiUWe9drpuyMtQY8wfgMnATKAP/uEzfYApwOvA\np+G8oLW2b6B/+KvUiERs3PLsuq8XZCXe5Lj7Pl1CcUU1pZXV3DF8kdvhOKq4vIpfD5nDP75YzrPj\n10TdXk5+GS98l8nanEL+78MMByJMPknUuSgx9EXGNvaX+O/m1SbqQF2N/kRRm6gD3PPJEhcjcdbO\n/MS8S5xKnEjWa3vOg3XRdW10XEA149Lfxz/c5WZr7Vprbam1di1wM/6hMNcbYy6IPuTkl0y36MQ5\nm/YU132dnV/mePtu/tmNXZZNYXkVAB/P3xp1e/oAE3FGeYIl5eFo/Bk7bPZm+o9ewa4C58+rXpWx\nZS9/+2QJE1fluB1K0nNiGExt5ZZgY9JrJ4sGG9Ne61KgLTA9wERVnzFmBtC35t+0yEIVERFJXbsL\nyvhkwTbOOPZAzunVw+1wksKcDbkMrJkkvyWvhA//3M/R9ofPzeKtaRu59dxjuOP8Xo62HY3fvDkX\ngK+XZYc4Mn6qqn1UVls6tkuu8r9O9Kyn12wvNcY0aM8YkwacC5QCoabDt6/ZHhxkf+3jGjyJes5F\nROItGc66fx+5jFcmr+N3784nt8ibk3MTzaglO+q+nrk+1/H2nxiziuz8Mp4dvzbhhg7FU25ROec9\nn86Zz0xm8dZ9bofjqKiTdWvtRmAicAz+ai71PQl0BoZba+vuvxtjTjTGNK7MMrNme50x5pT6O4wx\npwLX4T9XTo02ZpE5G3K56Z15fDg3y+1QRKSGE+sR7swvpahmOJSbvsjY7nYIAc3a8H0yOXWt+2U4\npWWqfErWg/n316vJKSijsLyKP7w33+1wHOXUCqZ3AXOAV40xFwNrgH7AhfiHvzzW6Pja2V9152Zr\n7QJjzFDgNmChMeYrYAv+i4BrgHbAYGvtKodilhT2u5o38txNeVxyUk96duvgckQiEq2pa3dx+weL\n6NyuDVMe+hmHpIX/vs4tKmfAmFV0ateaf1/9Yzq2a01JReRJ/6jFO0IflOC25pXQrWNbt8OQRqzL\n94DcuvG/btf3i6SVVFS7E0SMOJKsW2s3GmNOB/4N/AK4AtgJvAo8aa0Nt2zGn4EZwK3AZUAaUADM\nAt611oZVDSYVaSGZyG3dW6Jk3W1OdKlGG4IHYpDo/GmYv8JRYXkVT41bw2s3/STs5w4Yu4pvampM\n9+zWgQcvPYFPF2wL8azU9d3KHO4ckUHHtq05OK196CfEmfHCSUXEIU71rGOt3Ya/VzycYwO+i2oW\nRhpW809a4P3Zm90OQSSpGGXvIbndg9ecllbl+Gb594vBfJGxnQcvPSEpq5g45c4R/nKmpZXVbN1b\n4nI0IsktHnXWRZJGos/rVf4pqaS4vIq/j1zKPR8vZp+WUk8ZCX6aDmn0kh3c+M5cJq3e5XYoMVNS\nUUW1L9l/k+FzrGddRMRNiX4hlYi8/jP/75T1dWPH27Vuxcu/PdXliESiU1Hl4/7PlgIwb9NeR1Yb\n9VonzuwNudwxfBE90trzzb0/dWQ16kSnnnXxBI+dK2Jq8upd/ObNOY4s3CMSD/F6fzqd+3+64Pv3\nWP3yehKZwZPXcfWQ2czZ6Hx5Qq+bssadXuyMLQ2n/BXHYOKk1y66f//efIorqtmSV8LgSaGW6EkN\nulxJUB57b0kL3D7cPwkuY8s+fnnKYaqmICnBWsuugnLHJ3PrXBi+vCjqqq/ZWcDgyesB+N27yVUW\nL5Syymr+/MGioPtjeTFbu/BQqtqcWxz6oBSgnnWRFiitdLZXY6/G0UqKuHNEBmcNmsLAsaq+64Yx\nS3dw5rNTIn5+Zk5h6IOC8FrPbUvFYvGo/SUV7MwvbfHzGi+I+MqkdVGVGA1HVbUmWrtNybqkvHDH\n66Wv3c1HGrqSMpzqLcvKLebXb8zmz8MWUppktX/DlVtUzoRV/mEEw+ZkuRtMirrv06WasOeoyM8Q\nW/NK6PfsFM57Pp05G6IbUvTfKet5a/qmqNpozv9mbebHAyfQf/SKmL2GhKZkXSRMtw1b6HYIkoDu\n/ngxS7buZ8ra3byevt7tcFyRaEukuzmHJhlLhibhtxSVh79cRnmVj2qfrVugLxqvTondeeWpcasp\nq/QxYt7WiO4EiDOUrIukEK9+ZjoRV7QJwfLt+Q5E0dSq7IK6r6ev2xOT15AA1IkcteLyqpS9GxRL\nOfktWwOgpWK1IFRhWWyH27REog+tailNMJWUF+pNP3XtLj6a597wl535pdw5YjFtWhnevrkvPbp4\nb7XAZDB0dhaPXt6Hdm2i68PIyi3mtakb2JJXzEmHd22y31rLyh0FHHFgR7p3bhfVa4nEyursAm54\ney6tDIy95zy3wxFJaepZFwnhT8MWMWXtbtde/+EvlrNs234ytuzjya9XuxaH1znR07Jjf/S3ee/4\ncBFfLt7Ooi37GD53S5P9H87bwlWvz+Knz08lv6Qyqtd67tu1nPf81Aarb4o44Y4PF1FUXkVBmX9h\nKYneoG/XcNF/ppGVl/grvq7Kjs2dSAlMybpInOwrrmB0BLWeZ67/fgLSVJdq/Ur41u0qanb/E2P8\n1VCKK6qDjmEvKq9qUvWhsc25xbw1fSPb95Vy98eLIws2TMEiSbVb0V6zZmcBl70yg9uGLnB8XsD2\nfd9fuK4P8TcdSrL/nYQzBC8zp5C3p29iU5SlCL0yp+GO4Rluh5BSlKxLyovXue+ODxfVrTznJWOW\n7uBnL6bHdJJSIorH30VRedPxwN8s38lpT03il6/OorKZkmnZDtwFiBe38ouCGI6xfWliJte/NYeM\nLfscbTfURVp9t7y/gMxdhaRn7uG9WbGrCALenST65rSNMb9YdcKWPO/WC4/kd+vEXchoePXvMVaU\nrCeoZO+pCCQzp5AHRy7j62XZbocSkYVZzn6oO+W+T5eyJa+ElyetY3dhbCc+SWh3f7yYiiofq3cW\nNFiBU1oowId5fml4w45C9V6fEYRmAAAgAElEQVRmbNnHa1M3sDBrH795c04k0Tlid+H39b8Xbt7b\nzJHJad6mPJ7/bm3AYWDJmsy15GIu1XjlrkMsKFmXhHHjO3P5cvF2/vbJEpWQilB1iBN9qi/S5LUP\nwuwYV40IV7J8BF712ixHao0v27bfgWgkWukuziXyMhtk4Nq0zN0Mnb2ZonLvVHWR8ChZF08I5+Nz\nX73JeMu2aXJLS1VU+SirbDisIiuBlnIur6pm457oxs5Katu6t4T5m/PcDiOk0opqRszbomQ0RoIl\ns0GPD3i4Oxf2kfYnrN9VyK1DF/Lk16t5eeI6R9qU+FGyLpIiPlvYdEjFdW+5dwu/vlC3L6uqfVz2\nygwufmk6Q9I3xCkqCSXcpCdWdZ8jkQireL41fSP9R6/ktmELWblDHRO18ksqKatU3fdIvDl9Y93X\n78/e7GIkEgkl6yIOs9aybW+J54ZU1B/fWiu3yNvDXsqrqhk4dhUnDZhQV+7sxQmZLkclofh8ljU7\nC/C1MDF2NKX31tuvRf5bb7L3SxNj+Pceo5+Rz2cbJNUt7ckOZP6mPM58djJnD5riubk1HjvVR62q\nmYnt4g4l6yksv7SSQePXMCR9Q0L0NiWKez5Zwk9fSOeRL5c73nYyT6AJZOjsLIbNyYrbcvXxfxck\n5/vu/0ZkcPl/Z3LPJ7Gv0uG1i2KBC1+axhnPTGaBg5Neb3x3HuVVPvaVVDJw7KqQx8f+z8Kdc3E8\nPgLOez6dzQGGSIb7M3Xi4kwaUrKewv4zIZO3Z2zixQmZjFq83e1wXOPkua+0orquMsHIRYn3M/Va\n3vPx/PCroTj9AeGloRuJpKLKx6TV/vUAxq/IcTkaZ6XqhUFheRWDJ4df2nVLXgmFZVXc8PZcx2Ko\n/6PPyvXWokLJ1oeSU1DG3+JwoR1rydQJqWQ9SYXzmfLhvO9XV3xvZuqOYXPy7Ryq2orXvTtjE7M3\n5IY+MIjE/u4lmJYsiuSVXjUn4kikJCw9cw+FZdGtiNucQD2tXhXORVVOflnYwz0i/TsoLKtkxfb8\nhLzIW7mjwO0QWqzxz/n3781zKRLnKVlPWIn35hfvG7VkB79/bz7b9nqr58qL8ksryStqOg8g2Wzf\nV8IV/50ZVRtuJL17AszRaKnGOZbXh6G1pPc7nrx2l+qDOVs4a9AULv/vzJj0vr4xbQN/GraQkwdO\n5KrXZ2lSvEvmbdqbUBXPmqNkXTzBW6fyyG3ekxwnhpGLtrkdQos5nRCUVASvRbxxTxH9np3M2YOm\nsmSrNxe7cspDny+jNAErcCTRHfCw/W9W6t4hbYkva4Z9rt9dxLjlzi+y98J3mUytV3bzP41KJUYj\nFp30Hr8GjUpFkkyWVbIu4oD8kkpueHsuV70+y+1QEpJXbhPXD+PtGcGXb3/gs6WUVfqoqPZx27CF\ncYjMHSUVVczblHorY4qzIhmStG1vSYurCUViX4ovBBcLXruTkgyUrAvgnXGmieq579Y4WvkgGLdO\ngRNW5fDLV2fy3szgCaybRi7cxr++WuFom1ObWZBma71hQvtLYjdO2E2vTVnPyQMntug5g75dE6No\nvEVny9h6ccJafvpCOje8PbfZC/nmLqjle69PXc/ugsQcsueVVZzdpmRdUp4TCfC4ZTsdaCW2ovk+\n/+/DDFZlF/D0N2tiUuM42FjgcC8iH45BmcxU99KkdS0ez/v29OiTp9yicgaOXcU7MzaGdcfF6+PI\nBQrLWra8/ZB0/wI+i7bsY4UWhYrafyauY1YUhQPctGZnQUJNbo6VNm4HILFRXuWjY7vWbocRsY17\niuh1cJdmjlDfllt2F5RzSFoHt8NISI1zz1C5qBOjgyqqfLwxbQNllT7uueiHdGkf/9N+S9LpJ8as\nrCv5+IODOnPZj3o22D9+RXQXxh4ZcQX4S71+tWRHzNrPL63kfzM30bNbx5i9RijLt+/ntamRT7As\nqfD+fIlYDfsoDfC9p+K16aOjlvPpHWe7HYar1LOeoEJ94Jw1aArrdhXGJ5gYuPil6W6HIAnC2eXH\nPZTJOeSj+VsYPHk9b03fyGtTvFktpL76tdk/XdC0zv5dHyV+/edab07bEHL41uqdBTz59SoytrR8\nIvPz363l1amhXyOW/vzBItde2ymBPm9zi8p5Z8bGmL3mlrxi+j07ucnjsfhc99IFbCD5pS27M5OM\nlKx73Idzs+j37OQWl37KL63kryMywj7e62/WxsqrfHwwJ4uP52+N2dLI78zYyC8Gz2j2mFgvy+zk\nxMsE+xWHZca6PZz+dNMPtFQQ7p9G/XNH/TG++4orGDBmJS9PWpeUy4snwjycV8Pocd5VUM7Q2Vn8\n5s05VLbw99SSRcVixYkSmpGIRw/0s+PXkpkTm06xv49cRkGA4UN/GpY4Fz9OlcXcsa9pKeFA579k\nHhKnYTAe9/gY/7LKL07I5LZzj6FTO/+vLJy3wEaPlxHcklfMuOU7uexHh7b4uR/P38r8mgmdHdq2\n4trTjnQ0ttyicp4dvzbkcZcOnsGE+8939LXrm5a5J2ZtJ4M/vr8gbq91zD+/AWDZgEsjbmNVdtOF\nRiqrfbRtHf9+k2fHr+HzDH8Ju55dO/C7fke3uA3vp8NNeSXmr5e1vGRgQWklB3VpH4NoJFLhjqlv\naSK5PooedP+wHOf/0sO9AK6o9nHtG7PZureEIb87jX7HHRTV6wa6aEk16llPIOWV8e39mrR6F7cO\nXcDkmqXDnfbH9xfw4oRMrn1jTotrIs+vV3llQM0FjZPC7Q3atKeYkYu2xSwBGJwAwxaSS+gP1D8F\nKNU4dlk2+RFUhfl04TZ+PGACb02P3e30YGoTdYAP5mTF/fUDiUciHavzWUvkl1byt0+WxKz9ovIq\nPpybFbP2xdvcvqv0/qzNLN66n9yiCn77jjOriEZ6l/nFCZnsTYLynErWJSCfz/KX4YuYlrmH24cv\n4pKXp7M2x9nlh7fk+W9tFZRVkV+auOXv8ooS/0Qg4Qs0dvjeT5bwlw8juz1dXuXjuW9D38XxpHA/\nPz10d/rDeVvcDoEd+0ojel64vbMvTcysuysrLZeeuZv+o1ck9LyvlnB69EhViN63nPwyrhkym5EL\nw198L9LSyJNW72LA2MR/LyhZl4CqG13Frt9dxC8Gz+SSl6dzw1tzKSr37m2p3YVljFm6g8KyxL0A\nELeEl30G6uSJR519acorC2o5xYnvZujsLAdaSU35JZXcNnQhI+Zt5SaHeoW9yM3Px7MGTWHptv08\n/OVyVoY5jKgkikICkQw58xol6wKE/wGxfncRC7L28rKDyyc7yeez/Pbtedz36VIe+GyZ2+F4ioc6\nN5uIR2xlldX8ZfgirntzDlvyvD2fozmFZZWuTdrzqvySSu4Y3vDORjTl9Lw6T82jYcWdL4YXaGvq\n3UHOC3P4hFf/Xprzkkc+w1+Z5I04vE7JukRk3qa8mL/GixPWMi0z+CqSgWzKLa5bQGHyGvfHpqaa\nr5dlezaRHJK+gUmrd7Foy76YjheOpe37Suj37BTOHjSFuZsSc5GTWHjuuzVM9MBYdImPtSEqsCRz\nVRCnzFzvjeIFkV52pdqvWNVgxLP8q9htZO6jF3FY2It6JNct8UR0x4eL+Oquc6Nq46WJ/klB2/ZG\nNrY3kClrvr/wW749MVdF/NdXK+sWiald5TEUL4wSibSXO9zQv8xourBQNJPs9nl0QlqqJShe8J8J\nmczeGJ8L44/mb2FVdgF3X/jDuLyeJA4l6ylo296ShJrQOXXtbn7f7wdxfc2WDpNItnGzzQmVMCzZ\nuj/q14hmxcNa63cV8o8vlnPEAR0ZfOOpUbfnBYHqDUtg0bwliyuqm51Qn0rv92Q38OvVHH5ARy6t\nWSm38a/29RBrnGzOLXIkjsVb9/HYVysBWBej2u3JyOezZO4qpPehaW6HElOOJevGmCOBfwO/AA4C\ndgKjgSettSGXXjPGXACkh/FSR1trw59CLA1szi3m4pemNSmV6PaHj9ulpurbmlfCnSM8ukqiB3rW\nEiVPuX34IrbklbB0235+cvQBYT0nUb63QArLKunUrg2tW8XvjyTc9224x8Ur8oFjVzHgqpOaHS7x\nt4+XOL5+gxMqqnwMnb05ZMWNsHjgfBJKblF5zD+f7vgwg5VPXkaX9i1PicK/w9X89zB++c66rxdt\n2UfXDt7rS41mHkisPPj5Mr5asoPzex/sdigx5chfgzGmFzAHOAQYA6wFzgTuA35hjDnXWhtqkHMW\n8GSQfScD1wKrlKj7RXrueuyrFS2uae62WCdQjT+vn/w68cs8FZZVMmx2Fod0bc8Npx+VkmM4a0uD\nAszekPzju894ZjI9u3bg2/vOp2O71nF5zay82Pb01/7Vzlqfy7++WsGZx3Z3pN1hc7I445ju/PKU\nw4Iesz3C8oqxZDB8unArg5wq9ZkAnwX3fLyEkb23hz4wSrsKyuhycJeYv45XbNxTTEWVj3ZtWiV0\nJ8VXS/xD4Gas28Ph3Tq4HE3sOHXp9gb+RP1ea+1rtQ8aY14GHgCeAe5srgFrbRYwMNA+Y8wnNV++\n40CsKa3AoXJN8Xxv9x+9Mq6LGtSOCU4ECzbvZW1OAb/+yRGkdWhb9/jLk9bVlW87tGsHLjjhEJci\njI1hszfzaw/2erqprNJHVl4Jb03fyAOX9I7pa2XvLyWtQ5u4TSb+w//mA7B1r3MXB7M25DabrHvV\noDBWVk42M9a1bDKk23eKE8XQ2Zv5v5/1CrrfjcXaotG45HQyiboajDHmOOBS/D3jQxrtHgAUAzcb\nYzpH2P5BwK+BUuDDyCMV8OZtrHC8nKTlnXbsK+Ufny/jvZmbWv7c/aXc8PZcnhizihcnZDbYV7/O\n8hvTWn7C9fo5b+DXq3l63Gq3w/CkbY3GtQe6qbK/JLqL33s+XszIReH3dsZ6gmnKSMzTd0gz1+/h\n/VmbHetMioaXhmTGWnqIamvxXqxtX3EFXy0Jcl5JnV9LQE6UbryoZjvRWuurv8NaWwjMBjoBZ0XY\n/q1Ae+DzcMa+S/NScDSEp93/2RI+z9jO09+sYfaGXDbnFvPtip2UV4Xu3R82e3Pd18Pnur8qY7x9\nntGyW+Ph/O2H+3ng6Z67MEJ75ps1Ub3E4q37qfb5Qh/oolC/72+Wh14oRefL2Nu0p4ib/7eAf49b\nzYvfZYZ+Qoy4ufZCRZWPLxfHfqhPY8Xl/s+ZcP/OY30hc+M787Q+ShBOJOsn1GyDdX2ur9lGel/2\n9prt2+E+wRiTEegfcGKEMcTUqMXbueX9BcxxcVyth1OPkMKNPVivzbpdDWfzR3JCivTnl1v0fQ/n\nxwu2csV/Z/LXjxbzyqT1zTzLXbEYJhTrpCg9cw+rsoNX90gU4cw9yM5vONY60HXFuHqT2SL1bAyG\nYzR+L4Yjr6iCiuqmFw6hyn4WlFVRHWICT+Ofnacv0hLU/2Z93+nw4TznOh1aekr507BFWGtdufv8\n2cKt7CuJ/12FFWGuHhovmbtUBScYJ5L1bjXbYL/12sfDK8dQjzHmZ/gT7FXW2jkRxOZ5+aWV/H3k\nMqav28Pv3psf89dL/o6ipt/hx/O3csw/v+GUgRP5S6NVDgEeHbUiHoGF9M3ynZTWLKnc3FjBwrJK\nXpuyvsU9y055Y1pkZRXzisp5bcp6pq7V4jWxVJtsWGspTaD5F9B0IbNwzldPjFkZ8etVefzuQCCp\nNEwj3twqvvD4mMQvahCOUBfHElw8agPVnm8j+S3dUbMNu1cdwFrbN2Ag/t710yKII2Z2FZTF9wU9\nel93+bbIr/Ab9nY1/DNbuSOff331fTI+afUuMmNQwzbYT/XnL0/n4C7tef/WMxx7nZcmrmPYnCxH\n2ovEroLIJhU+MWYV36zw9+imP3QBx/aIaBqLY/48bGHEz/V6dZ1qn+W6t+awZmcBZZWJl5DWCudD\nY1mCLnAVCY//2cVVqPHWkdLFUGx8mbGdgV+v4ud9DnU7lITkRM967ZmyW5D9XRsdFxZjTHfgNyTB\nxNLyqmo+WbCVRVl7W/Q8p5cDLiyrZNm26BesiYXvVuVE/NziiuqgV+yNe+rA/3OIlw27i5i7KY/X\n050b1hIsUc8vrSQrN/bjLiPNF2oTdYAvMtyvwDplbWw+7L1g1OLtLNm6P2Cibq2tu4MTL0oynZOo\nRQKCibTCz21DI7/YDkbDnGLnwc+XUVhWVVdqsVZeUTkZW/bpZx+CEz3rtTNCgo1JP75m29JyHrfg\nn1j6gbXWmxlmGKy13PPxEiat3oUxMPPhCznywE5hPffm/y1ott2WKK+q5uSBE1v0nETS61/jue3c\nY7jpzKMbPO6VD7YFm1t2oRaJcwZNobiimtdu+kmDx73xE0gM0X5ehDMxOJj8kkq6dWob+sAQVu8s\nYO4XwZe1mJbpbCdAsvJiD2syvpdnrk/+NRAgvnfjEinvXbY9n9+8OYfHrujToucVlVXFKCJvcqJn\nvXbV0UuNMQ3aM8akAefi7x2f18J2/1KzTeja6ptzi5m02t+7ay3M3RhqbajYGLV4R/MHOPTmnrgq\nh4FjV8Wlh7exobOzGLu0YYWHQOfHiqrmhwXE4kQXj5Nncc345L99siT2L+agap+NaGKhF703c3Po\ng4LoN2gya3ZGPwk2v7T5O0exrnpRVZ1AmYJLVmanztCdhKM/X1c8M75lFarifXfQbVEn69bajcBE\n4Bjg7ka7nwQ6A8OttXWfEMaYE40xQSuzGGN+CvQBVib6xNLjGq2I1n/0SlcmWRSXt+wqNJI+gF0F\nZdzxYQbD5mTxpw+cv0UZjqlhDG0YMd/5MoehfqOLtjhTdbQgmt4Ej45F+Mfn3ivVNXpJiIvbIL6M\nYtJvWaWPuz5aHPHzveKSl6c71pY3/2Kj9/AXyyN6ns+X/ElKSz+rnFTloQmQUZ3rk5AX73TFk1MT\nTO8C5gCvGmMuBtYA/YAL8Q9/eazR8bWXUMHOxbUTSxO6Vz2Q8iof57+QztDbzqD3oWluhxOVsspq\nVu8s4NQjD6BVK8O8Td/fNdi0x72atfUF+gMbvyLy8fGBJNItx5aoqvbx9oxNTT48nf52R0WYGMdC\n7fXM/Z8tbfa4WI2v3OzCHSmnZec3nTQfadKdpG+tiI1Z5p33Sqz8aMAE11570updHNS5nWuv7xSP\n9svEXLJ+FoMzw2Bqe9dPB4bhT9IfBHoBrwJnW2vDHvthjDkQuI4kmFgazI79pdwRoIRgKPd8sphv\nV0RfH9kJ1lquGTKba9+Yw2Ojmy99uGN/KTf/bz5/+2QJZTHuFWr8Xk3Vk1a0jIGRi7bz4oTMZldA\nDXaXyKsTmUNJ5pO9BPfx/K1NHvNiFZ0nUqTEX7TSM/fw6tSWl5iN5V2LZJpA6eaaMPWNC2Nhs2Th\nWOlGa+024LYwjw2aQtWsUtrRqbi8Kiuv5TPgZ2/IY/aGPJYNuNTxeMqrfFRV+2jTOrzrt7X1yh9+\nsmAbg649pckxBWWVdO3Qlr9/tpT5NRMsex3cmdvOPZauHeJRNdT7Jfa8bEh66A+7r5cFPllePWQ2\nWc/90umQUk68b/3u2F/K54u20aFta0fb9XrlnfrlXetbvj0xLzrFe2Z5JMF1QjzWhAnHPR8n1vys\naMQnYxJH7dhXGvbYun3FFXw0fwtzQkxs3bG/lPOeT2f03efSs1sHJ8Jk4+4ifnL0gXWJOsDgyesZ\nkr6BHx4SmyFATqTmXun/cGpuQyTXK69N3cBhYfwdvDTJveXBxVmFZZWc+9zUmLQd6RjteKgMsPpp\nrW9XOjtkTrwtll07w+c6P1eqPmttzDun3Lw5EIuVsxOJI8NgJP7CfdMM/HoV/5m4LmSyDpBTUBa0\nh8lJldXWkaoX4Yjo3OWRbP357xxezr2FZ9qdAcYei7ti+VH8UYChIBJYLJN4L4yWcKOal0TnJ09N\nanbl60Q3bHaW2yG4Ssm6iyIdw5ZfWsnHC8K7Sh+ztGVjulYleEmxxsm5V+qsR+KdGZvcDiGg/SUV\nUff6p/qJ103B7spVNdPDLA1lOFTdyav+mgRViVLN/pJKnvvW4Q4eDylysUqQFyhZd9HirZGd8G96\nd57jFU2SRePrn3gNWfdCb1i87Mwv4+KXpoWcLPzx/K38/bOlbNjdtIZ6sUdvaaZCebDadR/Er6Xv\n3W0RrrjpRcEmVMbrzqfX7MwvS4EzgCQiJetx8IODAq9YGmwMVklFFeNdrvqSqCesJtVgXInCW+qS\nEQevXLLyShg+N6vZY/711QpGLdnBbcMWsLtQQ2q8orkx2qlmxfZ8zn8hPfSBNTbuKeanLTje61J9\nHHBjL09q6ULr3uPzUK14J9UvapGKlKzHQetWgZOkb5YHTsgf+XJFUiyO4obGPUKR5Kct7V3dtq/E\n0wuV1J/g66RdBeVhHbdtb2nS3Z6tDLJKZ3FFat+qTTS/e3c+OQXOXEhaa5m+bg9jlu6I6oJof2mF\nI/FIanozicetpzJVg4mD43p0CbhI0KcLt7Fjf2mTx4OVw0s0qVI28YsoVq2Ml7U5zt/W/t+szWEf\nO2pxci3mEuziLNwLGK9JkbdqExUO3mVYvHU/t7y/AAg8vva2oQsorqjmld+e2mw72/Y2/UyQ+Hlq\n3Gq3Q4jKixMyads6Nd/QyXweU896HNz+02OD7pu5vmHt1d0O9fKIXyJPMHVSVm5xUo21jZVnx69l\n1nrv1kN24sMo3hfRfx2REfFzM3MK+YNHajqH8q9R31fSeuyrlU32p2fuYcHmvTw4svnVcePJYvlu\n5U7uGL6owQrUqSzVh1s0Z2HW3ogmomvyevTUsx4HvQ7uEvaxf6zpmUk0ny7Yyj9HNSz76IUV28LN\nSyat3kXb1oaf9T44tgG1UIlDM+DvHKFhVeGYsW4PM9btcTuMoDzwlmqxaMoc7swvS7oSovM2xWZY\nWiS+W5nD8u3+CmATV+/SQmYp6Ia357IgzKGST4xZFVElpHjN80jE82O4lKzHwcFp7cM+NlGv6hsn\n6onmL8MXATDiz/1cjqSh9QEqqYjEgjq/Uk9toi6pK9xEvVZLy0GD1uxwgobBSFJr6S3/Oz5clNRX\n5yLBvDI58SthiEhy9zCnKvWsS0DJMtJ7056W9Uwny/ctEkyyL+gjkbPWRr3gmYg4T8m6xIwxxvWa\nr1pC3buqfZYVO3QbPpQV2/P5z8RMzul1kNuhSJI79tHxbocgIgEoWZeYuXXogoTsqc4vrXQ7hJRw\n90eL+W6VVuIN5TdvzqGi2sd0D098FRHvqErRuyPJXLpRybrEzP6SxEt6iyuqNakzTpSoh8fJWuAS\nG+mZu6nWQGERVyXzW1DJuoiISBRuG7rQ7RBEJImpGoyIiIiIJLTdhYm5gnQ4lKyLiKS4RF3fQZq6\nbWhiLqwnIsEpWRcREfGIFdv3R/X89ExNRBZJNkrW48Rry9gHs6ugnEpNaBMRccXAr1e7HYKIeIyS\n9Ti58Yyj3A4hbPd9usTtEEQEKKuodjsEEZGEV1aZ2OdSJetxctgBHd0OIWzjV6iknogXfDB3i9sh\niIgkvKGzs9wOISpK1kVEREQkab01faPbIURFyboENHH1LrdDEBEREYlaoq9MrmRdArrro8VuhyAi\nIiLiiJU78t0OIWJK1kVEREQkqc3blOd2CBFTsh4nxu0ARERERCThKFkXERERkaRW5bNuhxAxJetx\ncngClW4UERERSSajFm93O4SIKVmPk4PT2rsdgoiIiEhKWreryO0QIqZkPY6uPe0It0MQERERkQSi\nZD2Ozu3Vw+0QRERERCSBKFmPI59N3MkNIiIiIhJ/Stbj6MBO7dwOQUREREQSiJL1ODrsgA5uhyAi\nIiIiCUTJehz96PBubocgIiIiIgnEsWTdGHOkMeZ9Y0y2MabcGJNljBlsjDkwgrZONsYMN8Zsq2lr\ntzFmujHmj07FKyIiIiLidW2caMQY0wuYAxwCjAHWAmcC9wG/MMaca63NC7OtW4H3gBJgHJAFHAD8\nGLgCGO5EzCIiIiIiXudIsg68gT9Rv9da+1rtg8aYl4EHgGeAO0M1Yow5C3+ivhL4hbU2p9H+tg7F\nKyIiIiLieVEPgzHGHAdcir8HfEij3QOAYuBmY0znMJp7AWgN/KFxog5gra2MLloRERERkcThRM/6\nRTXbidZaX/0d1tpCY8xs/Mn8WcCUYI0YY44EfgosAlYZYy4E+gIWWAqkN25fRERERCSZOZGsn1Cz\nXRdk/3r8yXpvmknWgTPqHT8VuKDR/hXGmGuttRtCBWSMyQiy68RQzxURERER8QonqsHU1iPMD7K/\n9vEDQrRzSM32BqAPcG1N2z8EPgROBr4xxmhlIRERERFJCU5NMG2OqdnaEMe1rre93Vo7rub/BcaY\nW/An8KcDvwE+aa4ha23fgIH4e9xPCydoERERERG3OdGzXttzHmzFn66NjgtmX822HBhff4e11uIv\nCQn+kpAiIiIiIknPiWQ9s2bbO8j+42u2wca0N26nMMhE0tpkvmMLYhMRERERSVhOJOvpNdtLjTEN\n2jPGpAHnAqXAvBDtLAdygR7GmEMD7P9xzTYr8lBFRERERBJH1Mm6tXYjMBE4Bri70e4ngc7AcGtt\nce2DxpgTjTENKrNYa6uAt2v++0L9xN8YczJwK1AFfBFtzG669Zxj3A5BRERERBKEUxNM7wLmAK8a\nYy4G1gD9gAvxD395rNHxa2q2ptHjzwIXA38ETjbGTAMOxj+ptAPwYDilG72sZ7cObocgIiIiIgnC\niWEwtb3rpwPD8CfpDwK9gFeBs621eWG2U4I/WX8S6IS/p/5X+C8ErrDWvuxEvG5qfHUiIiIiIhKM\nY6UbrbXbgNvCPDZozlqTsA+s+SciIiIikrIc6VmX8LVprR+5iIiIiIRHmWOcdWzbOvRBIiIiIiIo\nWY+7U486wO0QRERERCRBKFmPs5MO7xr6IBERERERlKyLiIiIiHiWknUXHNW9o9shiIiIiEgCULLu\ngrf/cLrbIYiIiIhIAsL4g+gAACAASURBVFCy7gKNWxcRERGRcChZFxERERHxKCXrIiIiIiIepWRd\nRERERMSjlKyLiIiIiHiUknUREREREY9Ssi4iIiIi4lFK1kVEREREPErJuoiIiIiIRylZFxERERHx\nKCXrLrnwhIPdDkFEREREPE7JuoiIiIiIRylZd4l1OwARERER8Twl6yIiIiIiHqVkXURERETEo5Ss\ni4iIiIh4lJJ1l/zrij5uhyAiIiIiHqdk3SW9D01zOwQRERER8Tgl6yIiIiIiHqVk3UVtWxu3QxAR\nERERD1Oy7qL2bVq7HYKIiIiIeJiSdRepX11EREREmqNk3UW/PeMot0MQEREREQ9Tsu6i+y/p7XYI\nIiIiIuJhStZd1KV9G7dDEBEREREPU7IuIiIiIuJRStZFRERERDxKybqIiIiIiEcpWRcRERER8SjH\nknVjzJHGmPeNMdnGmHJjTJYxZrAx5sAWtDHNGGOb+dfBqXhFRERERLzOkXIkxphewBzgEGAMsBY4\nE7gP+IUx5lxrbV4LmnwyyONVUQUqIiIiIpJAnKod+Ab+RP1ea+1rtQ8aY14GHgCeAe4MtzFr7UCH\n4hIRERERSVhRD4MxxhwHXApkAUMa7R4AFAM3G2M6R/taIiIiIiKpxIme9YtqthOttb76O6y1hcaY\n2fiT+bOAKeE0aIz5LXAsUAGsAaZaa8sdiFVEREREJGE4McH0hJrtuiD719dse7egzU+BQcBLwHhg\nqzHmusjC87Zf/b/D3Q5BRERERDzKiWS9W802P8j+2scPCKOtMcBVwJFAR+BE/En7AcBnxpjLwwnI\nGJMR6F9Ne54y4KqT3A5BRERERDwqHnXWTc3WhjrQWvuKtXactXaHtbbMWptprf0X8CD+WJ+NZaBu\nOKhLe6497Qi3wxARERERD3IiWa/tOe8WZH/XRsdF4j38ZRtPNcakhTrYWts30D/8JSU955I+h7od\ngoiIiIh4kBPJembNNtiY9ONrtsHGtIdkrS0DCmv+q6oyIiIiIpISnEjW02u2lxpjGrRX0wt+LlAK\nzIv0BYwxJwAH4k/YcyNtR0REREQkkUSdrFtrNwITgWOAuxvtfhJ/T/hwa21x7YPGmBONMQ0mexpj\njjPGNBm8bYzpAQyt+e+n1tqkW8U05GB+EREREUlJTq1gehcwB3jVGHMx/tro/YAL8Q9/eazR8Wtq\ntqbeY+cD7xljpgMbgb3A0cAV+MfDLwIediheT7HK1kVEREQkAEeSdWvtRmPM6cC/gV/gT7B3Aq8C\nT1pr94bRTAYwAugLnIp/YmohsAIYCbxtra1wIl6vsepbFxEREZEAnOpZx1q7DbgtzGNNgMdWALc6\nFU8iUc+6iIiIiAQSjzrrEoJydREREREJRMm6B1h1rYuIiIhIAErWRUREREQ8Ssm6B6hjXUREREQC\nUbLuAaoGIyIiIiKBKFn3APWsi4iIiEggStY9QMm6iIiIiASiZN0DlKuLiIiISCBK1j1ApRtFRERE\nJBAl6x6gVF1EREREAlGy7gXK1kVEREQkACXrHqDSjSIiIiISiJJ1D9CQdREREREJRMm6B3Tt2Nbt\nEERERETEg5Sse8BlP+rJ8Yd0cTsMEREREfEYJese0LqV4dv7fsqkB853OxQRERER8RAl6x7RpnUr\njjiwo9thiIiIiIiHKFn3EINxOwQRERER8RAl6yIiIiIiHqVk3UOMOtZFREREpB4l6yIiIiIiHqVk\nXURERETEo5Ssi4iIiIh4lJJ1D9GYdRERERGpT8m6iIiIiIhHKVn3ENVZFxEREZH6lKyLiIiIiHiU\nknUPad1KPesiIiIi8j0l6x7SupXhxetOcTsMEREREfEIJesec/3pR7kdgoiIiIh4hJJ1D3riypPc\nDkFEREREPEDJugfdes4xbocgIiIiIh6gZN2DWmmiqYiIiIigZF1ERERExLOUrHvUjWdooqmIiIhI\nqlOy7lGPXt7H7RBERERExGWOJevGmCONMe8bY7KNMeXGmCxjzGBjzIFRtHm+MabaGGONMU87FWsi\n6NapLb885TC3wxARERERF7VxohFjTC9gDnAIMAZYC5wJ3Af8whhzrrU2r4VtpgEfACVAFyfiFBER\nERFJJE71rL+BP1G/11p7jbX2n9bai4BXgBOAZyJo879AN2CQQzGKiIiIiCSUqJN1Y8xxwKVAFjCk\n0e4BQDFwszGmcwvavBq4DbgXyI42xkR1xg8iHkEkIiIiIknAiZ71i2q2E621vvo7rLWFwGygE3BW\nOI0ZYw4B3gVGW2tHOBBfwvrDWT9wOwQRERERcZETyfoJNdt1Qfavr9n2DrO9d/DHdWc0QSWDNq1V\nrEdEREQklTkxwbRbzTY/yP7axw8I1ZAx5k/A1cBvrbW7Ig3IGJMRZNeJkbYpIiIiIhJv8ei6NTVb\n2+xBxhwDDAY+t9aOjHFMIiIiIiKe50TPem3Pebcg+7s2Oi6Y94FS4K5oA7LW9g30eE2P+2nRti8i\nIiIiEg9O9Kxn1myDjUk/vmYbbEx7rdPwl3/cU7MIkjXGWGBozf7Hah4bHV24IiIiIiKJwYme9fSa\n7aXGmFb1K8LULGx0Lv4e83kh2hmOv2pMY8cD5wNLgQxgSdQRi4iIiIgkgKiTdWvtRmPMRPy11u8G\nXqu3+0mgM/C2tba49kFjzIk1z11br517A7VvjLkVf7L+jbW2f7TxioiIiIgkCid61sE/znwO8Kox\n5mJgDdAPuBD/8JfHGh2/pmZrEBERERGRgBypBmOt3QicDgzDn6Q/CPQCXgXOttbmOfE6IiIiIiKp\nxKmeday124Dbwjw27B51a+0w/BcBIiIiIiIpRUtkioiIiIh4lJJ1ERH5/+3dd5hU1fkH8O87M9t7\nYXfZXlhYdhe2sssusMDSQQRFkSpVAVFjjdiiaBI19q6JGsUSa9RfEmOJiRoL1mg0sSP2aIIV7Hh+\nf8xdGGan3Jm5d+6d2e/neeYZmLnl7D1zZ9577jnvISIim2KwTkRERERkUwzWiYiIiIhsisE6ERER\nEZFNMVgnIiIiIrIpButERERERDbFYJ2IiIiIyKYYrBMRERER2RSDdSIiIiIim2KwTkRERERkUwzW\niYiIiIhsisE6EREREZFNMVgnIiIiIrIpButERERERDbFYJ2IiIiIyKYYrBMRERER2RSDdSIiIiIi\nm2KwTkRERERkUwzWiYiIiIhsisE6EREREZFNMVgnIiIiIrIpButERERERDbFYD2GLOwo2/Xv5rJs\nC0tCRERERNHgsroApN+42kGY3VQMANj85jY8/+5nFpeIiIiIiMzEYD2GCIDumnwAwOYtn1hbGCIi\nIiIyHbvBEBERERHZFIP1GKI8/j2uNt+ychARERFRdDBYj1GjKnOtLgIRERERmYzBegwrz021ughE\nREREZCIG60RERERENsVgPYYUZiZbXQQiIiIiiiIG6zZ33vwmZCS5MLupGG0VOZaUoak0y5L9EhER\nEQ10zLNuc/u2lmJOcwmcDun3Xk5qAt6JQrr16kHpeOG9z83fERERERHtgS3rMcBXoA4A585vgp+3\nDBWFXRARERGRDwzWY9iQggw88tOJuG1tF5IT3FXZUp5t/I4YrRMRERFZwrBgXURKReQaEflARL4V\nka0icoGI6O5oLSLHisg92rrbReQLEXlRRM4TkVKjyhpPSnNSMaoyFw8cOR7XLG/HbWu6cOXStqjt\n/+z9RkZtX0REREQDjSHBuojUAHgWwAoATwE4H8AWAD8B8ISI5Onc1BoAxQAeBnAZgKsBbANwJIB/\niUiLEeWNR2W5qeitK4TL6cCU4YW4Znk7Nq3sMH2/+7eXmb4PIiIiooHKqAGmlwEoAHC4UurivhdF\n5Dy4A+1fAFirYzuNSqlvvF8UkYMA/FrbzkxDShzHHA5Bb12h1cUgIiIioghF3LIuItUApgLYCuBS\nr7dPAbADwFIRSQu2LV+BuuZW7bk2zGJSmLqq9d4UISIiIiKjGdENpld7vl8p9aPnG0qpLwE8BiAV\nwOgI9jFbe/5nBNugMJwzvwnCEaZEREREljAiWB+mPb/m5/3XteehejcoIqtF5FQROUdE7gNwHYC3\nAWwIv5gUqvQkF0qyU6wuRtiiOdCWiIiIyAxG9Fnvm97S36w5fa+HklNwNYBOj/8/DWCRUuoNPSuL\nyLN+3qoLoQwDXqy3p09rKMLm4ydh9BkPWl0UIiIistBli1utLkLYopFnvS/mU3pXUEqNVkoJgHy4\n+8MDwLMiMt3owpF/uivMhs6aNwIAUJSVbHFJiIiIyGozRwy2ughhM6Jlva/lPMvP+5ley+mmlNoG\n4AEReRrAKwA2iUiFUurrIOv57P+gtbjH7qUV6bZvK9PyExERUewzomX9Ve3ZX5/0vgwu/vq0B6WU\n+gzAEwAGAWgIdzsUHonB/jAJTv0fbWa8ISIiIrsyIlj/m/Y8VUT22J6IZAAYA+BrAJsj3E+J9vxD\nhNshAgAcPWUo7ljXjaGF6T7fH1KQjgsOaI5yqYiIiIh2izhYV0q9CeB+AJUA1nu9vRFAGoBNSqkd\nfS+KSJ2I7DHYU0QqtJzt/YjIGgCjALwL4MVIy0zW+eNhY1E/ODP4glEwrbEIbRU5ft9vLM7E3JYS\nv+8TERERmc2oGUwPAfA4gItEZBKAl+HO5jIR7u4vJ3ot/7L27NnBogXA70XkcW2djwDkwZ2ffQSA\n7QCWKqV2GlTmAWdRZzluevIdS8vQWJJlm241WSkJVheBiIiIKCBDssForevtAK6FO0g/GkANgIsA\ndGkDRYN5DsD5ABIBzAJwDICFcCclORdAvVLqYSPKO1CtHFMV0vJKxXI+mOAKMwNnihG7XFUQERHR\ngGVUyzqUUu8CWKFz2X5RkFLqHbiDfDJJuLFnsNVmjRiMP734oenloOjoq584v1aLuq7qPDyxRU+7\nBRER0W7RyLNOFkpLdO76d0l2Cn67YhREgESXA2OG7JkFZX57eOkON85pwLja/IjKSfZhp0G1vz+k\n25L9zm4qNnR7I0uz8LuDR2PLL2fG9Llyp0X1Qbs9d/IUq4tARFHGYD3O3b6uG8u6KvC7g0YjOcGJ\nicMK8MixE/Hk8ZNQkp2yx7JnzRsZ1j7y05Nw/apO7K0zwJGYnxs1vg3KSLJNq3pruf8BwGa6eGGL\nodtzOtyfeYdDUJWfZui2o6nFovrw1loeyoTYsa9n6CCkJ7lw2eJW5KYlWl0cIooyButxbvjgTGyc\n04iumt2t6GW5qcjx8YXPPtr29/ovZhiynUBZcGJ66toQ1Rb4TtsZS6pjOPgP14zG2J2JMBybVnbg\nHz+bEtMzMJpt7JDYvWNFFAyDdfKrL2bLS0+ytBy0WyiTPQUyiz/6AIA5zcZ2d9HD6LsWJ8wcbuwG\nyZaMOvfjFduaKJ7x7Keg1k+s2XXrdUp9ocWl2VNyAj/C4ejrlmFnZ+47wrRtT60vxP5tpTiox+fU\nDobLNjFN6MS6AtO27YvR/fnDwcAsuDyLussc0F5m6PaKgmTtIhoIGOlQUBnJCXj0uIn4+08n4sol\nbWgqy4bLIWEHU0b+0D7/s6nGbWwA2f6t/ScCXtBRbtq2f31gO87evwlJLmfwhSOU4BScPrdx1/+N\n/PyfN78p6hdep+3dgA0z6nDFkrao7pdCc8ua0Zbs95cGX2SfOc+8i3aiWMFgnXRJTXShLDcVDofg\nrkO68exJU4IGU/cf2YPOqlzTynTk5KFITggebJXmpARdxpdohECJLv2n4BkG/QgmOh348POv/S+g\n8w8/aRa7X+jx2IZelOakmrLtFB2ff6PlpCVi7fgadHtlkyJ7GVKQYcl+jb54DPf725fKPHPOQyKz\nMVgfwIYVZYa1noggKzX4bf2hhRkYPrj/Poz6Kh9VpS8zxXUrOwK+v3pcdLpChGvJ6HLcsKrTsNvL\ndx86Jmi/6dTE4EFgepK+aRp+PrcRyQkOzBqpv5+8d6YiKxw5eagh2ynIMO82/rihg0zbdjDJUbgr\nQURkhPMPaLK6CBFhsD6AHdhVgY6qXOSnJ+KmgzqtLo5pagalB8wUUJabipsPtuaWsZ7MK0MLMzC2\nNh+OEFusagvS0eujP/PwwZnY+WOAHSvgqgPbd/13494NIe3X25LRFfjnKdNw6aJWjCzNimhb0TQo\nw5yB1UYNML1hVafuCyZvo6sjv+MVyl0hoki0B8peZSCnQwLeGTDivLGjc/aP7UA2mESXA/u0hDeP\njF3w23YAS3A6cOuaLjx5wmR01/QPZk3LtW3y6LDfrhgV8jqjq/Ow9cxZQZcrCDOAy0j2HVQpE/Mk\nXr6kDavHVu3x2oFdFQCAHwIF6wC6h+Tj7vVjcNf6Mf0mzwpHX2B31bL2IEu6BUwtSQCA+uLw7ozR\nwGV2t6nuGnO6Rl2wIDoTtT12XC+eOXGy3/fNysjzwJE9pmxXLzslagi3ASLe2aeGyDLRHqC2fkKN\n3/fmt5fihVOmhp1a8Jrl7Zg4zLzsGLObijG/vTSkFuIFo8rw1An+fwDMMqQgvd+lwKJO9ziDgC3r\n2sehqSwbzWXGTj6jp0vI0MJ0nDK73tD90sCzYUZdxNsYPjgTGckubArSlS5WNJh8gecwoSEmLcml\ne8xHsExFwcZZFWUlIyctERcuaNbVFdAotYXWjC8IxujMPnooE1oJ4yF5FIN1irop9YX4xT6NPt9L\nSXAiKyUB1YP6T/Ry1JTgfYjLc82dIEYA/Gq/JvzfoWN1t7LXFmYgJYpf/IHUaeMUclLtOQviHeu6\ncd8RPcztb7DGkui1wttlkiYjBhPec/hYPH3iZPRYODYgHDGQmTUol0Owf1spBmfpH7+yyEcwPmvk\nYMxrLcXy7kqsGFOpaztzmkvw79Om67rbGqlgXWsuW9yKjkpzu9/4i49HmZggIpriIdUrg3WKOhHB\n4s6KkNc7fFKtjm2HU6LwXL+qExOH+f8RT3Q5UFuQjqWjQ/9bzTa7KfJJkcI91glO/ys6xJyZdKN1\na3V5d6XpP6xGGGTioNe/HjNhj//bYbBwuEREV8Yp8q/vbl6oXto4DWeH2Jfa5eO7JS8tEefOb8Kp\nezcgyeXEr/YbGVZ5zHL2foH/xpLsFDh8RGpXLGnDQhPT25K9MFin2GVed28AQKaPiWw848hhRRn4\n7Qrft8dHlmbhmZMm474jekIeiGfkrJr+Wkxcvr79o+SOdd3oGToo7B/xcLSUG9udx5/kBCfGB7iA\ns4uRJdEb6Gtdq5ZENN6iOMu6yXjscnfCSuFcJOn5qO3fVorb1nbhmuX6xs+YrSw3vDtA0xuLsHHv\nBuRrdyFnNxXv0Q1Iz51oI3h3lbx+VYffMVpWkTjoCMNgnciPJB+Dbiry9P+IZiYnhJzBBQCyTJzt\nsk+gAMqpM7rydyEweXjgWW5HlmZj08oOrBvff+yC3lb1o0P8IfrlPqHnqO/yGCxnZOBm5MVYJA4J\nMHYkXE02y/ZzwKjwLwjPnR+dQY3e7lo/Bvu1x3bmCjvx/p4SEYyqzDW9y6RRKgNcuCW6HLhjXRd+\nNW8kfj63EafMrsec5mIs6izHwVGanfmu9WOwdHQFirOSccWSNoyrHYSnAwzS1cMOsyTbDYN12oPn\nSbJPa4mFJQmPv1BPT154PabUBw5EA/HVPULPWJpQwv28tEQkOh0RzS7Z7lXOlMQ9W0lay7NxyaIW\nv+ub3ZKamezCYV5dooJluSgMYcryuiL3YK+q/DScu38T5rWWYtMq4wYYtlfm4vS5jVgyOrI7C5Ee\nZjNuTN2wun8KWDOySk1vLNK1nN4LT1+Ks61pWW8uy8YYj+xc+Ry/YSnPi+todWmc2+xOZHD1snZk\npSQEHMNTkZeG+aPKkJWSgPz0JFy4oAW/3GdEVLtvnT63EY8fP2nXeRnpvs+f34Rb13TpmptDz90z\n9lmnuHPK7HpMHl6AaQ2FOG56aNkUjPhNdmrdM4zuY3zyrNCzi9Tkp/d7LZJz/oIFzboHOAUyr9V/\nq9uTJ0zC0ydO3vWl6S81pL8vr7vXj+mXHagkOwXTG9zbO2RCDX5/yBjsNdJ/y8eoyt1pFzNM6Cvu\nK3tRWpL/H4c1PdUhfVlfvXx36s95baU4d36T39kgvfP3693P0tEV+Plc+0+jvl+b/hbejqpcZCSb\ne1do1dgqXLO83bQZYT15Zza5cmnbrgs5szWVZePEmcMxraEQN/q4APLGVKfmOWV2A1aPrcLJe9VH\n1FgTivriTPxqvyZM0u5S/myv+j3SK95i1bwgUeJyOtBRlatr4rXVY/vfQYhWF6BoYrBOe8hPT8JV\ny0bhyqXthnTHqAghI0Oi04H1E9235g/sqkSOR2t4pEFuUVYymnSkIbx2xSjkpCZg7JB8zPMRqPjq\nx+7LbB/BbHF2Ck6ZvecEQ+VhZKzoGep/gieX06HrLsLwokyU++grme1n3cuXtOK5k6fgp0Eu4Ioy\nk7FiTBUuWtiCfVpKcMuarqBlCZdna3qRn24qWSkJIV10Zqcm6B4QecEBzbh0UavubVupLoTZis8/\noAnluak4espQnLN/066WxWDdZvzd3TCqVevwSbU4ea969NbpD5j0plitLeh/Ye49zf20hiLce0QP\nDhpX1W9ZMxzUU40rl7ZjWJALhOr8NNyxrjsqZQKAXy/1f9cuL92eWab02KfF953k3LREnLRXPVaN\nrTJl8LsehZnJeGLDJDx0zARsPXMWOquNzWdvp5bnpFBb5b3Kftz0un7JKGz054XNXqMAKO4s7qzA\n7c++h7f+twMXLvDfdeLgnmos7CjfdbsvJdGJR346EW/+dwdcDtGdHzjQl+kgHT8kE4YV4JmTpuzR\nenv72i7csPltzGku0XV7b2FHGZbrvLgwIpfvFUva8OeXPsSKMf2DiNRE36e4wyG4bW0XNm/Zhp/c\n/Pyu1/11FxER5KYler225zJ3rOtCbWEGEpwO7N1UjL1N6nfYV8fXrezAwt9shlMEFxzQjMnnPdJv\n2faKHDgcgp07/d/3qS1Ix+sfbwcANJX6v6C76sB2rN70DAD3IKpxtfYfSAoAZbkpOHHmcNz+7Hu6\nlt+npXSP2f4uXNCC0/ZuNKwrWbjCaS0Ld/DeIRNq/H6XFIWQSjAqAkQi16/qwNKrn9rjtdlNxfjD\nCx+EvbsJXvNYXL64FetufA4pCU6cOGu47u0kuRz49ocfwy6H0U6f6zudsBV8dR3LSUtETlrsXgzp\nFUkXTn864iAFJYN1MlWiy4E/HjYW33z/Y8Bc4711BajyGkiTkZwQcFIes5LBeHezaK/M7deP258H\njuwJOsHF8u5KXPv4VoyqzMHgrBS89P4XYZcVcPff9deHt7U8G02lWXjhvc/7tYwWZiZjTnMJhhVl\n4Jan38WMxsER9TVsq4juF2JHVS4eO64XCU7x26dTz2fk8iWtWPSbJ5HgdODMef67pvTWFeDaFaPg\ncjgC9pNMCiH7z52HdOOc+1/F01s/xXcmBS4PHzOx30BnBdXvfAskkkDdtJmQTZQWoPtWps0yXfRJ\nTXTiq+92AtjdjdDXBeVRU4aGHazPbiru17VuxojBePjYCchOSQzpc3LTQaMx7/LHfb7X1+3OTN7X\nYkZ3vRxSkI43tEYA2tM1y9vx+dff47jbX8R3O3d/7929foyuO+ChCiUxhF2xGwyZTkRsMymQ2fTc\nTjxldj3+clQPbj64y+zskxAR3LGuG387ZoLfLix1RZk4ZXZDTLY+FGUlRzyB0pCCDDy2oReP/HRi\nwAlYHA7BhGEFGFubH/AOzqLO8l2pyw7vHRJw3y3lObhx9eh+F1KBxiWEyl9GonmtpRhdnbtHX9hw\nZw62o0imUA90Ho/3miApULeQqNC+RK5atjsV4VXLRvlZOHwzGotwtp8c5RV5aYbeefnJ5OBzaviS\nbZPJ3pITHLboImfEGClvK33cwQ1Vb10h9mkp7ZcXP5xAXU8XFzt18wmXPZsIiOKYiPgdsAj4+PKJ\n8JvG5XSE1IpqJau+UxOcxrVbpCa68ODR4/HGR9sxOsy+pSfNGo7n3/0U27/9AROGFuCWZ941rHx9\nnA7BzQd34YedP+LtT77CU299gpmN5gfrh/cOwX3/+givfvQlAHdr7X0v/Qff7fwRB3ZVYNMTbxuy\nnx6TuikVZCbj2hWjsHnLJ1jaVWHYpE8Ogd95G/TorsnHPYePgwgwfLD/boNpYTac9NYVIDnBiW9/\n2BluEXULNwPOkIJ07NNSgjv/8b7BJdqtKDMZ//niG7/v1wxKw+/XjbG82xgAHD11GCrz0lBbmI5F\nv3nS73L+7n4prze2njkLd+jsTuePv3FRejSVZeOFdz+LaP+xisE6xZU4uICOGVZ1b/BXx3s3FeP/\nvG7vR/vzkKp1IyrISEZBBLOE5qQl4i9HjcfOHxVu2GxM8OqPy+lAzaB01AzqP8jSDMOKMnHU1GHY\ntv1bvPbRdnRW5cKx0F1TVz78piH7uGNdV1hzHOg1YVhBv77bkXr0uF4URxj41+sY21OQmWx6QBuu\nuqIM7NNSgkEZoQfrfRlxzj+gGVu37cA/3nEHdUbn7L7poE7c+Y/3MbW+CLMvebTf+5cubrVFoA64\nu/Ys6660uhjISU1AgtOBr7/fies8LkhD/Q25YVUHfvPIFlz01zdCWo+TIhF5sPvpYMe+s94tFxS+\n0+c24ox9o58O8fgZ7u5FGckurBhrXKYQEYHLwBZ/M3jmnV4T4iQseelJ6KrJMyWoTtKR8i0QI3/c\nj502TNdy4Qbq4XyDBBqb4Y/ZmVAuW9yKe4/owRofk6X50lSWjcN7h6A6Pw1jhuRhrcd6ly5qxeLO\ncpw+x/jufdWD0nH01GEY4SPT0G1ru0LKvGQVz/S60VCQkYzHNvTiyRMmRdQnPSM5AeVx0P88HGxZ\nJ6K4kJWSgIUd5Tj+9y9Gdb8H91SjszoPFbmphg9SA+B3vIe/1xeMKsPNT7u7zRwWpM98pI6dPgwF\nGUkYnJ2yx4yvUClQ2AAAGAxJREFUtNuijnKcfd+ru/6fk5qAT7/63sISRSbBEfkFpK+uODNDGC9R\nlZ+Gu9ePAQAcOWVovwuJ4uwU/CKMWYsjMamuAKOCJCKozEvF1m1fBd1WPPSx9pbgdPTrbmjE36nn\nIjIejqe9m22IAvCetASIj5MyVvBYu4kImsuyTUurNtejW0BtQTqay7Lxq3kj/WbumdpQiDP2HYHj\nptdhXZDc6JHKTE7AYZNqsV9badRzUAfql20nCV7Zga5ZbvzgTzP80k+w63AIrl/VgdlNxbhJx4RN\nvty21ri88FblPg/HrWvNm3ciWux+uKM1cVm0sWWdYsr89lLc+sx7KM1JsWX2Er2TJvVhJxgKJsnl\nxINHj8ebH29Hc1l20OBERLCwozzgMvHQ++qKJa0Yf/ZDPt+L9O+LZkBSnJWMQ3vDy35ihjvWdeG/\nX36HycML8NRb23DX8x8g0enAtIbdk1GNqx0U9jwDp8yuR31xJp59+1OjimyZ0dW52LzlEwDAXk3B\n7wxEMo7FSHbJmhOOTh+/+zeu7sSJd76IUZW5mDAsNua/CBWDdYopP587AjNHDEZzWbbPaed9tbab\n7ax5I3DO/a9hfnupKV/GuTH8xRqqwdnGHj+7twLplZmcgJZyTinvqSIvDWfuOwIbotztyWiPHz8p\novWNHvfiOV/Cxr0b0VaZi9bybGQk22PQpJ2cN78ZZ937CoqzUzCnqf8MqC3l2bsGulqtb/KqRKcD\nG/duwAP//ijoOlbftfDO6Q+4JzpLcAq+1ya6667JQ4LTgYeOneh3O/HwM8BuMBRTEl0OTBhWsEfL\nQN8EGsMHZ/abIjwaDhhVjqdOmIRjp+mf1j6Y0+Y07ErB5m/Co3hz6ux6nxc73jNXdg/Jj1aRoqbd\nI0AKZVIluzr/gKZd/z5vflOAJSMzr60UQwrSYWLil4jZuWzBZKUmYOnoCjQU9x9MSe6+8RcuaMFx\n0+t8DpS2U1etGSMG4y9HjcdjG/RlHWoqy0aWjzvFdmgAef5nU3HhgmY8etxEXWl362xUD+FiyzoZ\nxqo76xcubMbmLZ+grSLHspaAsPfr46CNrR2EKfWFmNE4GHlpiaamoLOT5X4m21g3oQav/udL/OnF\nD1EzKA2nzq6PcsnMN7Y2H2vH1+C5dz7FSSFM2W5XezeVICXBidREl6nd1RKcDtx3RA8+2fEdRv3i\nL7teL8uN7KLdyDOu7xg89dYn6K0zNt1jLGsqzdqVs3xqfWHwFeKQ0WklgxlSoD89602rO/H4m9sM\n2a+vFnIg/PMsLcmFOc3972R4uumgThxz6wtoKMnCvi2Bl40FDNYp5iW5nP1mFYxVa8ZXY/Jw9w96\nOLmGY0lJdgpKslPw/mdfoyNAFoUEpwOXLm7FJUpZflvWTBtmGHdnxmpOh2B6GBMs7dtaijP+/ErI\n+xqUkYQ71nXj6ke3YOaIwbbrk7tpZQeee+dTtFfk4l8ffB7x9kR298sPNOuur/XswuV04JY1o/HE\nm9tCvntooz8jIvu22jOILM1JQZoJma2iqbsmH49t6I2b34zYv99KFMO8WxyOnzE8Zr5cptbv/oHt\nu8AIhcMh+N1Bo3H6nAZcsrgl6PKxclwofO6guws/2yv0uydtFTm4bHEb9hoZvLXS6LsXuUEyASUn\nONFdk49Eg7o43bR6NFwOQXKCA2fNG2nINq1QkZeGBR3ltru4MkqsfGOV5abqXjacOQismpQonn4z\nGKwTUVhy0hJxx7puHD+jDmeGGTCU56ViaVelbbIkkPXaKnKx0sDJpbxddWA7lgeZ1THU3/hNKztQ\nkJGE2hC6GUSiqyYPj2/oxZMnTEZ5nv5Ayw44UNUeLl/cioKMJCzsKENzBBMVUXTE9n0OohCYlQd7\nIGuryNk1zbc9xU/LipHiIHNjWFwOwWQT+kc3lmTh8Q29cDoEVcffY/j2fSnIjJ0L3JNmDcfP//Qy\nynNTMac5uv20/UnxM0/BQDFjxGBMbyyytPX5ooUtWHXdMwCACxc0W1aOWMCWdRowNsyoQ6o2c55d\nvhiq8gfm1MlkHjNmUY1Hh07cPburd8ahcLicjn6BTzjdw8xkRX59pYDV46rx0DET8MBRPbqyd0RD\ncXbKroGta8ebM3lYuIc7Wt1GwgnUPQeMRzqQe+KwAlyxpA0XL2zR1X1tIOO3Og0Y+elJeHxDL7bt\n+A41g6JzuzqY9ROH4M8v/gcfffENLl/SZnVx4pbLIchNS8QnO76Ly4G7P5lUiwsffB3NZdlot/Wd\nDvs4ZGIN0pJcyEpJwLQGc9KjTjVpu7Go0oYNE1cubcN/t3/LbngA0hKd2PHdTgBAZZ7/uirOTsFl\ni1vx99f/h9XjIuuu5nDIgElNHCkG6zSgZKcm2mowU2qiC385ajy++WEnUhN5OppFRHDb2i78+cUP\nMXNE6FlK7O7IKUOxX1spirNT4mpQlZlSE11YN8F3i6pVA+IoukTE1EA9lj5Fm1Z1YNFvnkSiy4Ez\n540A4H/CrZkjBsfl96idGRYdiEgpgNMATAeQB+BDAHcB2KiUCjqvsIikAZgLYBaAVgBlAH4E8CqA\n3wG4WCn1nVHlJbILh0MYqEdBzaB0W03rbrRQMjpE25CCdLzx8XYAwKiqgdPyX5Zj3zoh8tRWkYsn\nT5iERJeDv0c2ZEjnMRGpAfAsgBUAngJwPoAtAH4C4AkRydOxmXEAbgAwDcBLAC6GO0gvAXAOgL+J\nCO9V0YDnctijz2csYCOzPVy9rB0rx1ThmuXttu1yMKwwY9e/JwwLf96Gyxe3IjctEXOai9FVo+en\nL/pi8c6BZzrPU/dusLAk8Ss7NZGBuk0ZVSuXASgAcLhS6uK+F0XkPABHAvgFgLVBtvEfAEsA3ObZ\ngi4iGQAeAtANYD2Acw0qM1FMmlxfgLy0RGzb8R0Wd5ZbXRyioCry0vAzm888++sD23DFw2+itTwH\ntR6Be6jskGUjmESXA+0VOXjm7aA3vW1j8ehypCU5kZ6UgHG1+VYXxxBDbDJ2iuwv4mBdRKoBTAWw\nFcClXm+fAuBgAEtF5Gil1A5/21FKPQ/geR+vfyki5wK4EcAEMFinAS7J5cSfDh+HF977LKIWQBpY\nPIOzKQN0evdAKvLScMa+xkwwZNdA3bNY163swOYt2/DRF9/ihDtftK5QOiW5nDhgVGw1TvgazH7r\nmi5c9ODrmFJfaOuua2QvRtxP79We71dK/ej5hlLqSwCPAUgFMDqCfXyvPf8QwTaI4kZRVjKmNRQh\nyTWwcwWTfhctbMGqsVW4ZFGLbbIhRZtNY2hLpCW5MGl44a50tmS8g3uqUZiZBIcAFxzgThfcUZWL\nG1Z3YlmQibmIPBnRDWaY9vyan/dfh7vlfSiAB8Pcx0rt+V49C4vIs37eqgtz/0REMa04OwUn72Xv\nrihE8SQ10YVHfjoRn+74HkVZ+sdqNJRkmliq2HHIhCH4/XPvAwDW9FRbXBprGRGsZ2nPn/t5v+/1\nsOazFZFD4c4w8zyAa8LZBkXHmp5q/OGFDwAg6HTeRNHAhtT4EOnkK0RWSXI5UZQV/O7FbWu7cNof\n/o2Oqlx019ijT/7QCMZuGGFIQTpuWNWJt/63Hfu2llpaFqtFY9hv3+9lyJN5ici+AC6Ae/DpPKXU\n90FWce9IKZ+zy2gt7q2hloP0aSzJwq+XtuHtbV9hQUeZ1cUhojixeuzAblWj+DeqMhd/OGys1cXY\nQ2V+Gk6aNRx/efkjHDk58ll+wzG2Nh9j42RAcSSMCNb7Ws6z/Lyf6bWcLiIyF8DNAD4GMFEptSW8\n4lE0ccY+IjJacgLTlRJZYfW4aqweF9nFcnoy00FGyohvwFe1Z3+XXX2zkPjr096PiOwP4DYAHwEY\nr5R6NcgqREQUpxqK/bUFEZEdXbHE3YlBBDhrnjFZlgYyIy53/qY9TxURh2dGGC1H+hgAXwPYrGdj\nIrIIwCYA74Mt6kQUorREJ3Z8txMA0FQW1lAZsoE71nXjV/e+gnG1+WgsYbBOFEumNw7GHw8bi6yU\nBKaoNEDELetKqTcB3A+gEu5JizxtBJAGYJNnjnURqRORfplZRGQZgOsBvAOgh4H6wJGVkmB1EShO\n3HTQaJTlpmDMkDysHldldXEoTG0VObhlTRcO7a0NvjAR2U5jSRYDdYMY1ZHoEACPA7hIRCYBeBlA\nJ4CJcHd/OdFr+Ze1513JGkRkItzZXhxwt9av8DGxxGdKqQsMKjNZ7MSZw3Hmva9gUl0Bhg9mqioy\nRlNZNh45dqJtJ6YhiqbirGR88Pk3AID2ylyLS0NE4TAkWFdKvSki7QBOgzvN4kwAHwK4CMBGpdQn\nOjZTgd0t/Sv9LPM23NlhKA4c1FONhZ3lSE/i4BMyFgN1IrdrVozCITc+h0HpSTjcgrsUwwoH5gRc\nREYyLEpSSr0LYIXOZfv9kiqlrgVwrVHlodjAQJ2IyDx1RZl48KjxUb2AvemgThz+u3+gZlA6Fo+u\niNp+ieIVIyUiIqIocTqif9cn2neaumvy8dQJk+Gw4G8likdMXktERHHLM15MdgWfSdJsjcVZqMxz\nD7qb01xscWnMw0CdyDhsWSciorj12xUdWHbNUwCAXx/YbnFp3EHsnYeMwbNvf8qZGYlIFwbrREQU\nt3pq83H72i44HYJmm+Tdz0lLxOT6QquLAYCzwxLFAgbrREQUt0SEKQsDmDy8ECXZKXj/s6+xpiey\naeWJyBwM1omIiAYol9OB+47swesffWmbOw9EtCcG60RERANYepILLeU5VheDiPxgZzUiIiIiIpti\nsE5EREREZFMM1omIiIiIbIrBOhERERGRTTFYJyIiIiKyKQbrREREREQ2xWCdiIiIiMimGKwTERER\nEdkUg3UiIiIiIptisE5EREREZFMM1omIiIiIbIrBOhERERGRTTFYJyIiIiKyKQbrREREREQ2xWCd\niIiIiMimGKwTEREREdmUKKWsLkPUiMi2lJSU3OHDh1tdFCIiIiKKYy+//DK+/vrrT5RSeZFsZ6AF\n628ByASw1YLd12nPr1iwbwod6yu2sL5iC+sr9rDOYgvryx4qAXyhlKqKZCMDKli3kog8CwBKqTar\ny0LBsb5iC+srtrC+Yg/rLLawvuIL+6wTEREREdkUg3UiIiIiIptisE5EREREZFMM1omIiIiIbIrB\nOhERERGRTTEbDBERERGRTbFlnYiIiIjIphisExERERHZFIN1IiIiIiKbYrBORERERGRTDNaJiIiI\niGyKwToRERERkU0xWCciIiIisikG6yYTkVIRuUZEPhCRb0Vkq4hcICI5VpctHmjHU/l5/MfPOt0i\nco+IfCIiX4nIP0XkCBFxBtjPXiLykIh8LiLbReRJEVkWpGzLROQpbfnPtfX3ivRvtjsR2U9ELhaR\nv4vIF1pd3BBkHVvWiYg4tXL8U0S+1sp3j4h0Bz8SsSOUOhORygDnnBKRmwPsx/TjLyIpIrJRRF4V\nkW9E5GMRuVVEhod2VOxJRPJEZLWI3Ckib2jH5XMReVREVomIz991nmPWCLW+eH6RT0opPkx6AKgB\n8BEABeAuAGcC+Kv2/1cA5Fldxlh/ANgK4DMAp/p4HONj+TkAfgCwHcDVAM7W6kIBuM3PPg7V3v8f\ngEsBnA/gXe21c/ysc472/rva8pcC2Ka9dqjVx83kOnle+zu/BPCy9u8bAixvyzoBIABu8zhfz9bK\nt10r7xyrj7UVdQagUnv/eT/n3X5WHX8ASQAe1dZ5GsBZAG4C8D2AHQA6rT7WBtTVWu3v+wDAjQDO\nAHAN3N+DCsDt0CY89FiH51iM1BfPLz581q/VBYjnB4D7tA/1YV6vn6e9foXVZYz1B9zB+lady2YC\n+BjAtwDaPV5PBvC4VicLvNapBPCN9qVX6fF6DoA3tHW6vNbp1l5/A0CO17a2adurDOXvjKUHgIkA\narUfgwkIHPjZtk4ALNTWeQxAssfro7Tyfgwgw+rjbUGdVWrvXxvC9qNy/AEcr61zGwCHx+tztNf/\n5fl6LD4A9AKY7f13ACgC8I72d87zeJ3nWGzVF88vPvrXsdUFiNcHgGrtw/uWj5M0A+6r1x0A0qwu\nayw/EFqwvlKrk+t8vNervfew1+unaa9v1Ls9AJu011f4WMfv9uLxgeCBn23rBMAj2usTfazjd3ux\n/tBRZ+EEE6Yff7gvNN7WXq/ysY7f7cXLA8AJ2t94scdrPMds+vBTXzy/+Oj3YJ918/Rqz/crpX70\nfEMp9SXcV7OpAEZHu2BxKElElojICSLyExGZ6KcfZl+d3OvjvUcAfAWgW0SSdK7zZ69lIllnoLJl\nnWj769b2//cQ9jOQFIvIGu28WyMiIwMsG43jXwOgHMBrSqm3dK4Tb77Xnn/weI3nmH35qq8+PL9o\nF5fVBYhjw7Tn1/y8/zqAqQCGAngwKiWKX0UArvd67S0RWaGUetjjNb91opT6QUTeAtAA912Rl3Ws\n86GI7ABQKiKpSqmvRCQNQAmA7UqpD32U9XXteaieP2wAsGudDAHgBLBFKeXrh5T1CEzRHruIyEMA\nliml3vF4LVrHX893rvc6cUNEXAAO1P7rGbTxHLOhAPXVh+cX7cKWdfNkac+f+3m/7/XsKJQlnv0W\nwCS4A/Y0ACMAXAn3rcQ/i0iTx7Lh1InedbK8nlnv+ti1TliP/n0F4HQAbXD3Yc4BMB7A3+DuQvOg\nFkD0idbxH+h1diaARgD3KKXu83id55g9+asvnl/UD4N164j2rCwtRYxTSm1USv1VKfWRUuorpdRL\nSqm1cA/iTYF79Lxe4dRJuPXIetfHrnUyYM9fpdTHSqmfKaWeU0p9pj0egftO4ZNwt9qtDmfTISwb\nzc+F7YnI4QCOhjurx9JQV9eeeY5FSaD64vlFvjBYN493y4O3TK/lyFhXaM89Hq+FUyd61/lC5/LB\nWicGGrvWCc/fEGm306/S/hvKeWfU8R+QdSYi6wFcCODfcA/u+8RrEZ5jNqKjvnzi+TWwMVg3z6va\ns7/+W7Xas7/+XxSZj7Vnz9uFfutE6z9YBfdAny061xmsbf89pdRXAKCU2gHgfQDp2vveWO97smud\nvAFgJ4BqrRx61iHgv9rzrvMuisd/wH3nisgRAC4B8BLcgZ+vieB4jtmEzvoKhOfXAMVg3Tx/056n\n+pihLAPAGABfA9gc7YINEF3as+cP0F+15+k+lu+BOzvP40qpb3WuM8NrmUjWGahsWSfa/h7X9j8u\nhP0MdH3ZrbZ4vR6N4/8m3Hmrh4pIlc51YpaIHAf35DfPwx34fexnUZ5jNhBCfQXC82ugsjp3ZDw/\nwEmRzD6+DQByfbxeAffIdAXgBI/XM+FumQhlcpAqcFKkSOpoAoJPimTLOoG+SUMyrT7GFtRZJ4BE\nH6/3asdRAei24vhjgEzaAuBk7e95Bj6+A72W5TkWW/XF84uPfg/RDjSZQERq4P4yLABwN9xpsTrh\nni3wNbhPuG3WlTC2icipADbAfRfjLbinSq8BMAvuH6J7AOyjlPrOY525cE/v/A2AmwF8AmBvuNNS\n3Q5gvvI6KUTkMAAXwf2ldwuA7wDsB6AUwLlKqWN8lO1cAEcBeE/bbiKAAwDkwX3xdokRx8COtGM8\nV/tvEYBpcLcE9eXz/Z/nMbNrnYiIALhV2+4rAP6gLXsA3J+veUqpu0M7OvYUSp1p6eMaADwE97EE\ngJHYnV/5ZKXUz33sw/Tjr+WP/ivcwcszcKfFLQewP9yfkV6l1JO6D4wNicgyANfC3Y3hYvjuI7xV\nKXWtxzo8xywSan3x/CKfrL5aiPcHgDK40wt+CPeH+W24B5cEvLrmQ9exHQ/gd3B/0XwG9wQT/wXw\nANz5a8XPemPgDuQ/hbsr0osAjgTgDLCv2QAehvuCYAeAp+HOdxuofMu05XZo6z0MYC+rj1sU6uVU\nuFtZ/D22xkqdwD0XxZFaeb7WyncPvFq2Yv0RSp0BWAXgj3DPHrwd7la4d+AO6MZZffzhzgK1Ee67\na99q3wm3Aai3+jhHqa4UgId8rMdzLAbqi+cXH74ebFknIiIiIrIpDjAlIiIiIrIpButERERERDbF\nYJ2IiIiIyKYYrBMRERER2RSDdSIiIiIim2KwTkRERERkUwzWiYiIiIhsisE6EREREZFNMVgnIiIi\nIrIpButERERERDbFYJ2IiIiIyKYYrBMRERER2RSDdSIiIiIim2KwTkRERERkUwzWiYiIiIhsisE6\nEREREZFNMVgnIiIiIrKp/weFkMhunSmheAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1aa101b710>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 250,
       "width": 373
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(losses['train'], label='Training loss')\n",
    "plt.legend()\n",
    "_ = plt.ylim()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAusAAAH0CAYAAACEkWPuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3XmYFNXZNvD7gAoKuKDx1U+TuESU\nNzEvwTUSNWokxiTGLZtRo9EkRhOXaGISo2JU3JUgoqggIi64sIPIvg7rzAADzLAMzDD7vu/Tfb4/\nenqc6enqrr1OVd8/L68euqurnqquOvXUqVPnCCkliIiIiIhIPf28DoCIiIiIiOJjsk5EREREpCgm\n60REREREimKyTkRERESkKCbrRERERESKYrJORERERKQoJutERERERIpisk5EREREpCgm60RERERE\nimKyTkRERESkKCbrRERERESKYrJORERERKQoJutERERERIpisk5EREREpCgm60REREREimKyTkRE\nRESkqEO8DsBNQogDAI4EkOdxKEREREQUbKcAqJdSnmplJimVrAM48vDDDx86fPjwoV4HQkRERETB\nlZ2djZaWFsvzSbVkPW/48OFD09PTvY6DiIiIiALsnHPOQUZGRp7V+bDNOhERERGRopisExEREREp\nisk6EREREZGimKwTERERESmKyToRERERkaKYrBMRERERKYrJOhERERGRolKtn3UiIiIKgHA4jOrq\najQ0NKCtrQ1SSq9DogATQmDAgAEYMmQIhg4din793Kvvtm1JQoiThRBThBDFQog2IUSeEGKcEOIY\ng/O5TgixXAhRK4RoFUJkCyEeE0IMtCtWIiIi8q9wOIyCggJUVFSgtbWViTo5TkqJ1tZWVFRUoKCg\nAOFw2LVl21KzLoQ4HUAagOMBzAGQA+B8APcBuEoIMUpKWaVjPk8C+DeARgCfAagC8D0ATwAYLYS4\nUkppfdxWIiIi8q3q6mo0NzfjkEMOwQknnIBBgwa5WtNJqSccDqOpqQmlpaVobm5GdXU1jjvuOFeW\nbdeePRGRRP1eKeW1Usp/SCkvB/AKgDMBPJ1sBkKI7wB4BEAtgP+TUt4mpXwQwIUAJgAYBeBhm+Il\nIiIin2poaAAAnHDCCRgyZAgTdXJcv379MGTIEJxwwgkAvtwHXVm21RkIIU4DMBpAHoDXYj5+HEAT\ngFuEEIOSzOo6AALA21LK/dE3ZeTe1r8ASAB/EkL0txozERER+VdbWxsAYNCgZKkFkb2i+1x0H3SD\nHZeil3e9LpZS9mrAI6VsALAOwBGI1JAnckLX6/7YD7rmU4lI7f3ZlqIlIiIiX4u2UWeNOrlNCAEA\nrj4nYUeb9TO7XvdofL4XkZr3YQCWJZhPZdfrqbEfCCGGAIg2DDoLwNZEAQkh0jU+OivR94iIiIiI\ntESTdTfZcUl6VNdrncbn0fePTjKf+V2vdwohTon57ClEmsgAgKHeZYiIiIiI/MqNftajSXbC+wVS\nyjQhxCQAfwSwXQjxGYBqRB4sPQ/ATgDfBBBKtkAp5TlxA4nUuI/UHzqR/0kpPakJICIiIuvsqFmP\n1pwfpfH5kTHTaZJS3gXgDgC7APwCwF0A2gH8EEBW12TlpiMlSjEPf7odF4xdhkU7SrwOhYiIKK5z\nzz0XgwcP9joMZdmRrO/ueh2m8fkZXa9abdp7kVJOkVJeKKUc1PX/JVLKpQC+2zXJZguxEqWM9Pwa\nzNhSgPKGNtw1PcPrcIiIyEZCCEP/T5061dF4GhsbIYTAT37yE0eXk4rsaAazout1tBCiX88eYboe\nDB0FoAXABrMLEEKMBvB1AKuklEVWgiVKFQXVzV6HQEREDnn88cf7vDdu3DjU1dXhvvvuw9FH935U\ncMSIEW6FRjaznKxLKXOFEIsR6fHlHgCv9vj4CQCDAEySUjZF3xRCnNX13Zye8xJCHCmlrI9573QA\nbyLSVv0fVuMlIiIi8rsxY8b0eW/q1Kmoq6vD/fffj1NOOcX1mMgZdnVQejcibcnHCyFmCyGeEUIs\nB/AAIs1fHomZPrvr/1iThRCbhBCvCyHGCiE+BrADwMkA7pRSmq6dJyIiIiKgoqICDz30EM4880wM\nHDgQxxxzDH74wx9i5cqVfaZtaWnBiy++iBEjRuDoo4/GoEGDcOqpp+L666/H6tWrAQATJkzAkCFD\nAAALFizo1fzmxRdfNB1nKBTC+PHjMXLkSAwaNAiDBw/GhRdeiClTpsSdftmyZfjRj36Ek046CQMG\nDMCJJ56IUaNG4bnnnus1XXFxMe677z4MGzYMRxxxBI455hgMHz4cd9xxBwoKCkzH6xRbeoPpql0/\nF8B/AFwF4GoAJQDGA3hCSlmtc1bzAfwBkYdLhyByAfAZgOellNvtiJWIiIgoVe3ZsweXX345ioqK\ncNlll+HHP/4x6uvrMXfuXFxxxRV47733cNNNN3VP/8tf/hLz5s3Dd77zHdx2220YMGAAioqKsHr1\naixfvhyXXHIJzj//fPzzn//EM888gzPOOKPX9y+66CJTcYbDYdxwww2YM2cOTj31VPzxj39EKBTC\nzJkzcccdd2DDhg148803u6f/7LPPcOONN+LYY4/FNddcgxNOOAGVlZXYtWsXJk2ahIcffhgAUF9f\njwsuuADFxcUYPXo0rr32WnR0dCA/Px+ffvopbrnlFnz1q181uXWdYVvXjVLKAgC365w2bj9yUsp3\nAbxrV0xERERE9KXf/OY3KC0txZw5c3DNNdd0v19VVYVRo0bhrrvuwtVXX42jjz4aJSUlmDdvHi65\n5BKsXLmyVzfAUkpUV0fqYs8//3z87//+L5555hkMGzYsbhMdoyZPnow5c+bgoosuwtKlS3H44YcD\nAJ588klcdNFFeOutt/CTn/ykex2iifuGDRvwjW98o9e8Kisru/9esGABCgsL8e9//xtPPvlkr+la\nW1vR2dlpOXa7udHPOhEREZFrTvnHAq9D0C3v2R+7tqx169Zhy5YtuO2223ol6gBw7LHH4tFHH8XN\nN9+MuXPn4tZbb+3+bMCAAX3G6xBC4Nhjj3Us1mhTlxdeeKE7UQeAI488Ek8//TSuvfZavP32273W\nQwiBgQMH9pnXcccd1+e9nvOMivddFTBZJyIiIkoB69evBxBpsx6v9ruoKNLhXnZ25LHCE088EZdd\ndhmWLFmCc889F9dddx0uvvhinH/++Y4ntpmZmRg4cCC++93v9vns8ssv754m6je/+Q0WL16MESNG\n4Je//CUuu+wyjBo1CieeeGKv71555ZX4yle+gkcffRRpaWn40Y9+hFGjRuHb3/42+vWz61FOezFZ\nJyKilLKjqA6ZB2twzYiTcNThh3odDpFrqqqqAESagixYoH33obGxsfvvuXPnYuzYsZgxYwb+/e9/\nAwCOOOII/OpXv8ILL7yAoUOH2h5na2sr2tracMopp8QdgXvIkCEYNGgQamtru9+79dZbMXjwYIwb\nNw6TJk3CxIkTAQAXXnghnn32WVx66aUAIrXsGzduxJgxYzB//vzu7fA///M/uPfee/Hwww+jf//+\ntq+TFUzWiQJKQnodApFy6po7cM2EtQhLILOgFi//gn1PB5GbTUv85KijIoPNT548Gb/73e90fWfw\n4MEYO3Ysxo4di/z8fKxatQqTJ0/GlClTUFxcjM8//9z2OAcOHIgBAwagrKws7ueNjY1oamrCSSed\n1Ov966+/Htdffz0aGhqwYcMGzJ07F5MmTcLVV1+NrKwsnHbaaQCAU089Fe+++y7C4TB27NiBZcuW\nYcKECXjkkUfQv3//7odRVaFmfT8RETki42ANrnhpJe5+Px2hcOpd0M3dXozoas/M4Bh7lFouvPBC\nAMCaNWtMff/rX/86br31VixbtgwnnXQSFi9ejJaWFgDoro0OhUK2xDpixAi0tLRg48aNfT5bvnw5\nAGDkyJFxvztkyBBceeWVePXVV/HAAw+gubkZS5Ys6TNdv3798O1vfxsPPPAA5s+fDwCYPXu2LfHb\nick6UUAJxO10iVLcL95Yj9yKJizMKsWn6er1J0xEzrn00ksxcuRITJ8+HR9++GHcaTIzM1FTUwMg\n0h95RkZGn2kaGhrQ1NSEww47rDtJP/zww3H44Yfj4MGDtsQarfn/+9//jra2tl7LjjbHueOOO7rf\nX7JkSa/poqK180cccQQAYOvWrSgsLEw6nUrYDIaIKIV09qhN31pQh1+e52EwROQqIQQ++eQTXHHF\nFbjpppvw0ksv4bzzzsORRx6JgoICZGZmIicnB1lZWTjmmGOwf/9+XHzxxTj77LMxYsQInHTSSait\nrcW8efNQW1uLf/3rXzjssMO653/FFVdg/vz5uOGGG3D22WfjkEMOwQ9+8IPuGn0j7rzzTsybNw/z\n58/Ht771LVxzzTXd/awXFBTgd7/7HX72s591T/+nP/0JNTU1uPTSS3HKKaegf//+2LhxI9asWYNh\nw4bhuuuuAwDMnz8fjz/+OL73ve/hzDPPxHHHHYf8/HzMmTMH/fv3x0MPPWR9Q9uMyToRERFRijjt\ntNOQmZmJ//73v5g1axamTZsGKSVOPPFEfPOb38Tf/va37n7KzzrrLDz22GNYuXIlli5diqqqKhx7\n7LEYPnw4xo0bhxtvvLHXvN944w3cf//9WLlyJWbPno1wOIyBAweaStb79euHWbNmYcKECXj33Xfx\n+uuvQwiBb37zm3jsscd61aoDwOOPP4558+YhIyMDixcvRv/+/fG1r30NY8aMwV/+8hcMHjwYAHDN\nNdegoqICa9aswcyZM9HY2IgTTzwRP/3pT/Hggw/i3HPPNbllnSOkTJ02i0KI9JEjR45MT0/3OhQi\nx83KLMQDM7Z1/5sPXBHQu//pX5//NTxz/dkeRuO+9zbk49HZO7r/zePCn6JdCw4fPtzjSCgV6d3/\nzjnnHGRkZGRIKc+xsjy2WSciIiIiUhSTdSIiIiIiRTFZJwoo9gZDRETkf0zWiYiIiIgUxWSdKKA4\ngikREZH/MVknIiIiIlIUk3UiIiIiIh286PKcyTpRQPEBU0pGcBchnxJdO284HPY4Eko10WRduFiA\nMlknCii2WadkUmhMPAqYAQMGAACampo8joRSTXSfi+6DbmCyTkRERL4yZMgQAEBpaSkaGhoQDoc9\naZ5AqUFKiXA4jIaGBpSWlgL4ch90wyGuLYmIiIjIBkOHDkVTUxOam5tRWFjodTiUYo444ggMHTrU\nteUxWSciIiJf6devH7761a+iuroaDQ0NaGtrY806OUoIgQEDBmDIkCEYOnQo+vVzr3EKk3WigOID\nppQMHzAlP+vXrx+OO+44HHfccV6HQuQotlkn3ZraOpGWW4nOEJ++9wM+YEpEROR/TNZJFyklrpu4\nDje9tRF//2y71+EQkQ3YaoCCpKG1A+Ewd2oKHibrpMuuknrsKWsEAMzMKPI4GiIioi99sbMU5z61\nFKPHrUZbZ8jrcIhsxWSddOkMsbaCiIjU9Mf30tHWGca+8ka8sy7P63CIbMVknSig+IApEaWiopoW\nr0MgshWTdaKA4gOmlAx7g6EgYtlHQcNknYiIiIhIUUzWiYhSFHuDISJSH5N1IiIiIiJFMVknIiIi\nIlIUk3UiohTFB0wpiNi8i4KGyToRERERkaKYrBMRERERKYrJOhFRimJzASIi9TFZJyIiIiJSFJN1\nIqIUxQdMKYh4w4iCxrZkXQhxshBiihCiWAjRJoTIE0KME0IcY3A+3xNCzOn6fqsQ4qAQYqEQ4iq7\nYiUiIiIi8gNbknUhxOkA0gHcDmATgFcA7AdwH4D1Qohjdc7nTwDWALii6/UVAKsAXArgcyHEI3bE\nS0RERETkB3bVrE8EcDyAe6WU10op/yGlvByRZPtMAE8nm4EQ4lAAzwBoBXCOlPIWKeU/pZS3ADgX\nQBuAR4QQA2yKmYiIiIhsVFbfirfX7Me+8gavQwkMy8m6EOI0AKMB5AF4LebjxwE0AbhFCDEoyayG\nAjgKwB4p5e6eH0gpswHsAXA4gMFWYyYiIiIi+939fgaeWpCNG99Yj45Q2OtwAsGOmvXLu14XSyl7\n/SpSygYA6wAcAeDCJPMpB1ABYJgQ4oyeHwghhgE4A8BWKWWVDTETERFRALFLUm+l59cAAGqbO3Cg\nssnjaILhEBvmcWbX6x6Nz/ciUvM+DMAyrZlIKaUQ4h4A0wGkCyFmASgGcBKA6wDsBPArPQEJIdI1\nPjpLz/eJiIiIiFRgR7J+VNdrncbn0fePTjYjKeUnQohiAB8CuLXHR2UA3kHkoVUiIiIiDaxaV8X6\n3Cqc/pXB6N+P/cRa4UY/69FfKOnRI4S4GcBSRHqCGY5I85nhiNTITwDwkZ4FSinPifc/gBwzK0Dk\nR7wVTEREXnp87k48OX+X12H4nh3JerTm/CiNz4+MmS6urnbpUxBp7nKLlDJHStkipcwBcAsiXUP+\nXAjxfeshExERUTCxFlclU9PyvA7B9+xI1qM9twzT+Dz6sKhWm/ao0QAOBbAqzoOqYQCru/55jpkg\nVSGlRDjMKk9yHkenJKLUxHMsBYsdyfqKrtfRQohe8xNCDAEwCkALgA1J5hPtP/0rGp9H3283E6QK\nKhvbcNW4NbjspZXYX9HodTiGMPEjIiIicp/lZF1KmQtgMYBTANwT8/ETAAYBmCal7O6/RwhxlhAi\ntmeWNV2vNwohvt3zAyHECAA3InK5vNxqzF4ZM3cndpc1IL+qGfd8kOl1OIaw/TMREfkBz1cUNHb0\nBgMAdwNIAzBeCHEFgGwAFwC4DJHmL4/ETJ/d9dpdXyul3CSEeAfA7QA2d3XdmI/IRcC1AA4DME5K\nudOmmF238UB199/ZJfUeRkKpgCcsIiIi/7MlWZdS5gohzgXwHwBXAbgaQAmA8QCekFJWJ/p+D3cg\n0jb9NgA/BDAEQD2AtQDeklLq6g1GVUyeiIgoFUgpITxqP8lmmxQ0dtWsQ0pZgEituJ5p4x5KUkoJ\nYGrX/0RkAU9YlAx3EXLC2r2VeOiTbfj2yUfhjZvPQT+X+9hmxRgFjRv9rFMAMPEjCh7mNOSEmydv\nRGl9KxbvKsO87cVeh0Pke0zWSRfWVBARkVF7yhpcXybPVxQ0TNaJAoonLCIiIv9jsk5ERESBwWab\nFDRM1l3lr6pOKSUOVjVDsoqWiIh8gqcsChom66TpgRlbcckLK3D/jK2sqfAh/mZERET+x2TdVf7K\nnmZvjTzFP2drMcI6ayo6QmE8MGMrfjFpPXIrGh2MjpJh7RIRpSLps7vYRMkwWSdbTVufj1mZRdh0\noBp/mp7udThEROQh4bNKKiIVMVknW63cXd79954y1qwTEZG7eIFAQWPbCKakh39vzbHoIzvUtXTg\nxS92Y+Ch/fDg6DMx8ND+XodERA7yokkKm8FQ0DBZp7hie4Bh0ec/Kj5g+tLi3XhvQz4A4JhBh+Hu\n73/D44hSW3QXya9qwpCBh2LooMM8jYeIiPpiMxiFdYTCqGvp8DoMAH2Td1Kfij/ZtPX53X9PWXvA\nw0gIiFyEL9lVhktfWInvPrMMxbUtXodEAeNFkxQVyz4iK5isK6q+tQOXPL8C5z29FCt6tAMn51U3\ntXuStORXNeHJ+buwIoe/N7nn99O2AADaOsN4bM5Oj6MhIqJYTNYVNX7pXpTUtaK9M4zb39nsdTgp\nI7+qCRc+swzfe2451u6tdHXZv5+2BZPXHsDtUzejvKHV1WV7Q8F2OimuqqnN6xCIiCgGk3VFHaxu\n9nT5qXob8R+fZaG9M4ywBG6evNHVZffsPWfzgRpXl02UMlK1cCMi32Ky7iIj54j+/Vjr6IWy+uDU\naKv4gCmphbsIEZH6mKwrqp9iyTrrotxlR9djrEAkioNXsYHHoo+Chsm6ovordkJh4kcUPDysyS7t\nneG47yt2KiPyJSbrivK6GQxP4t7iCHxE/tURCmPVngrUNLV7HYrjpJS4+e2NGPGfxZi/vTjO5x4E\nZdDesgYs2lGiecFB5DUm64pSrTZCtXi89PrKXIx6djk+3HTQ61B8jfsUBdXjc3fit1M24cfj16Az\nFOwEcNGOUqzdV4nm9hD+/EGm1+EYVtHQhqvHr8Fd0zPwxqpcr8MhiovJuqJUawajlx9qUaLqmjuw\no6iu94BPSTZ7U1snnluUg6LaFvxzZpazAVrk012IXOTXXUT1Qdo+2Bi5kC+ua8Wafe52Aeu2wprE\nY1KoXg69vjIXHaHI/vTykj0Jp12fW4V7PsjA8pwyR2MKhSU2HahGQ6sagyKS95isu8jI6cXrZjCx\nFD83GtbY1omLn1+On7y6FlPW5en+XnN7yLmgbBa034wIAHIrGvGDl1fh2tfWKTPCcyKhUHAPxFBY\nolTBHrSMlH2hsP47H79+awMWbC/B76ZucbTJzH/m7cQvJq3H1ePXIBQO7v5D+jFZV5TXvcH0rbkK\nVoExZe0B1Ld2AgCenL/L42hSQyq03/U7taoI4rvn/QzkVjRha0Etnv082+twUpaUEte+tg6T1x7w\nOhRLzJ7Zmto6bY2jp3fX5wMACqpbsDbgd2ZIHybrivJrMxi/aDRZ0NrRpaJKy+np480FeGzODhTX\nJr6tbcaE5Xsx8qklvd7jHu49lS/BWztCuP+jTNz+ziaU1H25T+aUNnT/vT63yovQCMDqvZXIKqrz\nOozA6+BDrwQm68pSrBUMmFoFW1ZhHf7+2XZMW5+P+z/aavv8X1y8h81yyJDXV+Zi9tZirNhdgb9/\nut3rcChGvQ+aIOnBMxv5wSFeB0DxCeVq1plpAe51qeh2143zs77scm1TXrWryybvqFbK9NSzG8A1\ne603BSirb8WDH29jswLqxeyZjWdEchNr1hXl9QOmfVqsxymZlueU4ffTtmDF7nJXYrKTykkK4E0z\nGLcpdz3qoqzCOvzwldW4+/10PkDmkn/OzGKiniJSofyk1MKadUV5nawnEw5L/G7qFgDAkl1lyHv2\nxx5HZIxWUZ5sq/MkQHb49Vsb0NjWid1lDRj1jYP4zQVf9zqkwFue479KBVKX2mdoChrWrCuqn+LV\njiGNBshMZu3BEUyDrecDzun5NR5GQkREqmOyrijFK9Z9/7Cg2c3rVhLNix4iInWxhCY3MVlXlNfN\nYJxIxkNh6eqIbC0ODGAU1CSaNfkeCuYuZVlQNoviN0kdl+Krb1lQjgMzthXUIrei0eswlMBkXVGq\nFXCxBYbRE1BLewhXvLQS5z29FMuynR2qGQAenb0D3xrzBV5avNvxZREFhXq9UPmf3+9CWuXJ6ruw\nUB4pzlq0oxQ/e20drnhpFXJK670Ox3NM1l3Ud1RQ/zK6Kq+vykVeVTNaO8K4490tzgTVpaG1A+9t\nyEcoLPHq8n22zjsoNdDlDa14b30eDlY1exZDULalZQptBpXKKIU2C1Efbh0pqXoc3DU9vfvvhz7Z\n5mEkamBvMOSKgmr3ksJ2jviW1J/fz8SmvGqcfMx+rP7bZV6HQ9SHnmSIdwLUp/ovpND1KWlobrO/\nSavfsGadXOFmgW2l7E128g9Km/XowEeFNS0orW/1OBpShd+SX5XuBGjx2SYlxai/h5MbmKyryuMS\nPllSGpSklSKYUHiIh1Jc3CXJLCOHlNmyj/vnl9bnVmHC8r2oaGjzOpTAYjMY0iW2AuvCsct0TecF\nJwvRILaz9uon4wUCJcJmMOQGs+csBU51SihvaMWv39oAIDJmxDu3n2//QniYs2ZdVartm7G3m2ua\nOxJ+3oeLK+RkIco7CkRk1baCWny46WCvwbHIfxpaO5C2rxIdIeeek1ItF4j1xc4ve3dbsbvCmYXw\ntGtfsi6EOFkIMUUIUSyEaBNC5AkhxgkhjtH5/e8LIaSO/79qV8xu8/P+prcGixVdlMqklDhQ2eSL\nttSqC2pRUtnYhmsnrsM/Z2bhxS/YtaxfSSlx01sbcdPbG3Hvh5nd72cerMFNb23Aayvs6YmMJYm2\nwppmvLPuAAprvOvVzC22NIMRQpwOIA3A8QDmAMgBcD6A+wBcJYQYJaWsSjKbPABPaHx2NoDrAeyU\nUhbYETMlZvrWoF9KlqBmAj345rcIkD9/mIkF20tw7Yj/h3G/+o7ryw+FZdwB1VbklGNzXjVu+e7X\nXY/JrKDuvtM35Hcfm1PT8jDmmm96G5BJun+fgNbg7C5tQFZRHQDg8x2l3e9fNzENAJCWW4VLh30F\n3zrpKE/ic4srv26chUgp8dspm5Bb0YTpG/Kx7MHvuxGJZ+xqsz4RkUT9Xinlq9E3hRAvA3gAwNMA\n7ko0AyllHoAx8T4TQnzY9eebNsSqpPnbi/HBxoO47aJTMPqbJ3gdTh/Jagql9EeZXFDdjH/OzMLa\nfZVeh6IMKaUn1y4+2F0M6QiFsWB7CQBg9tZi15P1mRmFeHzOTlw+/Hj8t8eyi2tbcPvUzQAibUqJ\nXOVBrYEbd7YKa1qSTpNVVIdvnXQUpJSobmrHsYMH2Lb85vZOzMoswmnHDcZ3Tz/Wtvka5eVFdW5F\nU6/XILPcDEYIcRqA0YjUjL8W8/HjAJoA3CKEGGRy/scCuA5AC4D3zEeqrs5QGH/+IBNpuVX4w3uR\ngQBUS3z1HpC7SxscjcOq+2dsTZioK7bZLVFtHwq6sMe3Mv768TY0tHViztbiXkn50h4jBm88UO1F\naNRDUB5SV3ktWjpCjifsHWH97dRvnbIJ5zy1FBOW7+31fltnCCV1yZP+eMYv24dHZu3Ar9/agLzK\n4Cerqc6ONuuXd70ullL22nullA0A1gE4AsCFJud/G4ABAD6RUgayWqjdwYdT3PRuWh6qmtrjfqbK\nCSpZzWJQb70HVWFNM15evBub85iE9lRax77zKXV9sbMMN76xHqGw9yX6jqI6rNkbqSB6cfGe7vdb\n2kO49PmVuOjZ5XhjVS7ufj8dD368DS3t+gYAemNVbvffE1faO1K3EV6d2WOvxYL+nJAdyfqZXa97\nND6PXkoOMzn/O7teJ5n8Prnk8bk7NT9jLa89pJRIz6/Gxv1VgS+c9PjDtHSMX74PP39jPRpaO5J/\nIQUZ2U14mGrbV96Ae97PwNtr9jsy/wXbS/DPmVnIrWh0ZP6eSVD4z8woxDUT1uKz9ELbF5ueX4M5\nW4tsn69R9Rrl0pR1B1Ba3wo5HDxEAAAgAElEQVQpgWc/z8HCrFJ8llGICSv2xp2eUpsdbdajT0/U\naXweff9oozMWQlwK4CxEHixNM/C9dI2PzjIagxtUqXW2Iihpo5O/xDvrDmB9bhUeuHIYhp94ZMJp\ntRKsjQeq8as3I33avnPbebjsrOMNzyPedH69mNpVUt/999aCWlx8xlc8jIaC7LdTNqOotgULskpw\n7ilDMeKrhk9pAOIfa0W1LbjngwwAwLp9lVj998ushKqWBAXRXz/eBgB48JNtuH7kSbb3m1+ko125\nV93xVjbGH0DoUwcuXIIo9lfz83lMDzf6WY9uPjNHxB+6XgNdqx6vsHAigW/vDKM8BYaWV7HCeVdx\nPZ6YtwuLd5Xhpq4BJMy4a/qX16HRhwbt4MU244A27uBmtkdR7ZeJ39q99vYnvSH3y87SDlYHvxs6\nIPjNFoBI8qN1Lk+B1bcNizB7atajNeda/RMdGTOdLkKIoQBugIkHS6WU52jMMx3ASCPzCorWjhCu\neGkVSupa8MKN/4cbzjnZ1eWn+sG28cCXJ+PYAaXi0Uqw2jr0P9/AJI2AxElB7D7C/EEfK4mWCofl\n3rIGvL/xIEb/7//gom8c5+zCNAoiVZJVsxVjqsTvNa/OM6lwsdeTHTXr0VEdtNqkn9H1qtWmXctv\nEXmw9GMpZa2ZwPwiXmFh9wHwbloeimpbEJaRW47JGD0OUu3AMcr49nQmDi1+Suwb2zrxxc7SPm1B\ni2tb+GBlF72/p9n9rNWF3jZUZmTNVTy2fvnmBkxNy8NNb29Ek8FRVOduK8YLX+SgSqMZh17xmjF4\ngaNSB0PQf0U7atZXdL2OFkL069kjjBBiCIBRiNSOG733//uu18D2re6m8gaLBWvQj4QusSfWtH2V\nztc8kSF/mLYFablV+M7XercZfvizLPTvJzD3z6Pwzf/n/kAkqj57YiRZ1DPp4p2luH/GVpz+lcGY\nefdFOLS/G60p/UtP2el28Vrdo9euA5VNugfu2VFU12u0TitS+WIPsHf9e84q82ANJizfh8uHH4/f\nXOCfQdCMSrW9x3IpK6XMBbAYwCkA7on5+AkAgwBMk1J2dwQqhDhLCKH5sKcQ4mIAwwHsMPJgqepS\nuWxSsXZJj5snb7RlPlo/fXN7/FotIYDJaw/gxtfTsHavuQGc/Ly/zdtWjJ+/kYa524r7fJbW1b43\n82DfG26hsMRfbEomjFK1hs7u/eAP76WjuT2ErKI6fLjpoL0zN8iPxUq8stAvieuMzRxA3C5av7jV\ni/7rJqZhWU45Hpm1AwUp8vwD4J9jyCy7qkTuBlAOYLwQYrYQ4hkhxHJERi/dA+CRmOmzu/7XEn2w\nlLXqPhHUw6RPN70mVzReQXL3++k4e8xivLW6b1dwxbUteHL+LmzJr7HtgsFP/vJhJjbn1eDeDzMN\nF8Ll9dbuItkl6CcPAMiv8jYZ8GoLG+oOU0fuFfw9pS+bilalJfrt7SwetJaT3aO3LK912jyeTAoU\nr73Ykqx31a6fC2AqgAsAPAjgdADjAXxXSlml/e3ehBDHALgRAR6xNFa8A83rGiNVawmDYk9ZAxZm\nlSIUlnh6Yd/r1gMaI9IZ+V38ejeD3MN9hLzi92RLT/hedyeoUo9bM7aYvyujZz18vjslZUebdQCA\nlLIAwO06p9Xc8l2jlB5uV1wUYbVg3F9pbaAON9vzOnqhYXI1Yrd/rY4eYdwU+/uEwhL9++lbWSkl\n/jVrBzIP1uDOi0/DDTb3l9zzhNeho3ZGnRH1mAwHlZEyJna/UCmBMspM6FpfcaNCSOUEzo31d2NP\n03tuf2TWDlvb0KdahSKfDCJdHpm1I+HnVi8Gmto68cHGg9jCYeMB2FPrZGUeF4xdhg379d0QW5Zd\njg83HUROaQMe+mQbnpi3y/yCkbj5yMwMDhhiJ6v7mde1o/5Ne2MEOO/QWrVUGC7erWYwFHxM1hUU\nxEIrmVeW7MG/ZmXhxjfWo7g2+ahziajYK0dsLUCyGqrYXcDu9n6JlgVERteLjpSazLbC3g96Tk3L\nsyGq+B7+LCv5ROr9/OQQP5SUPq5I7yNAq+I5zQdMY8c+SMF8QI94dzKDjMm6i7w86IzeMjIaam1L\ne/KJEnh77YEv/15zIMGUyal4e8zqT//xFnVrlA/pZ64Y0d0XuKm5e09v3H5dv1Rm5Hguqk3e97+K\nZVZTWydmbD6I7YXWhjnRbAajyCqbjkPnF41e4JiJR5VtSc5hsq6AILR13VeeuE27X9bHjlr5/RWN\nqGnqffFitSwtqjXW60ZDawfe25Cve/rMghqjIXU7pL9aP64X0TR1DdTkN345LoHIQEyxPHs+Ifoq\nJdo6+8bV06OzEzchjMzHhqBs9vKSPXj4syxcNzENFRbH6Ygn9gJFlU1Q29yOyWsPID3f2SaZen9z\nK884uHF8q1KGqHjBaycm62QPF7qh6rNIKfHbKZtwyj8W4Oa3N3bfuXC0GUyS9Zy3rRiXv7QK3312\nGcobvqxRMzyCqcUN+vyi3dhaoL9GbN0+3R029XGIzgdR9bLaltXKyW13aQPueT/D0IUOAPzp/Qzc\n99HWXu/pjVuRcx0AoKa5He9tyEduhbUHyp3w5w8yvA6hl85QGNe/noZznlyKpbvKdH9PleQmmcld\ndztDYYlp6/NMz0dvm3VVPD53J56cvws3vL4elRZHaU0s/gYw9RCvxnf8sq8lE5DVsITJeopwumBc\ntafC2QXEsWRXWfdy1+6rxOc7IjWbXl5hRwfkae0I45mFOd3vO90MKZaRZNPq9tLba4xbrJygfvP2\nBizIKsGjs3dgT1mD7u+t9mD/d0JuRRMenb0Dv5y0Hu2dzj0nYcbS7HKvQ+jlw00HkXmwFo1tnbhz\n2havwzFtQVaJJ8t1o5Q2U47O2frlQGxzt/YdlM0uemPTc9Gv6oWPXfR2kxlkTNYVoOLtQKMxTIoz\nsE9Pdg95DgCbDvS+Tbk8x4WTuYH1qG6y1o7fL8wON+9Um3Urlw6VjV/+ZhsP+LdnIr3bQOsEV9nY\nrqutctBvPWuSEnk2Dgjl5VZ8fWWu7mnN3LXSbrMec97z2a60p0zf3SetbebG+qrY2YIedS3Juzb2\n2/5iFZN1RWkd4PvKG7FoR6lytV5afjtlE1raE7fpNMtPt/j69LXsTRiOsLvNelDKYL8/YKpqXCqQ\nMH8M+zWBspve/WvSqlzc+e5mT0bjDEuJA5VNcWu3jTaVc5KnAy/ZPL+FWSU476mlji7Dj5isu0j3\nyVvjkrGqsQ1X/3cN7pqebqg2xEur9lRgwoq9jsw79oLGlSttlzIYPyVKh5rsDQaI7OsPfrwNV768\nCps1+tg3+rv6edAZt3FTmWfntku1WkK9thXU4pnPc7A0uxw3v73R9eU/tSAbl724Ev9KMs6IlkQX\nZlp3pWK/Yak88+Hxfff7GWjX0VVxqt3VY7LuI5NW7+/eiV9ZusfjaPRbu68KRkoNvWWT6uVQz6LE\nqW47t+RVY962Ykt3WqyGZrbNuoDAFztL8VlGIfaWN+Lnb6y3Fohi/JCANbVp3/XSE78f1tEpto7S\nm2KJB6Bv31nfY2C2Kg+bFX646WDSado7w5ixOfl0UW60WffiHPn8opzkE5FhTNYV0Kf3C43pOkPu\nFehe9gmv+yToQEmUdNEml+nU5rzxjfX4y4eZeH+jd7dkrTSDySqq6/Oe1X3Prt1C9YtBO8zKLPI6\nBF+S0kIzmFTYsfToc97re9yHLZYFbl4Ezdh8MO6gbVq/d1Avzya6dNefgyKRsqwU8p4m347N14Oz\nnoHNaCU6oyepJ+btsrA0a8wOiqRXKtY6kg84WPzklNYn7b/d7/Qc1yolYMnOoY/O2elSJH1pd93I\nK8OgYLLukrL6VjS0duqa1mwBtbWgFt9/YQVuf2cTOhwcnt4MV8uMmO1XXp98BEGnxf6kQSpDlRsU\nSZFw/H6RUdPcjn/OzMLYhdmGm1mNXZiNS55fgUU7LAwUpfjms7OyILbMv2rcGvx4/FqEw4pvBB00\na5YVWTW3w4iem/Wu/+ytRXjxi90JexdTZVu6KdVWmcm6Cw5WNWPUs8sNfcdMwvGLSeuRV9WMFbsr\nMF2hJ9WN0rvqsdtIKzn63vMrHBvcQu8dC7sKUycKZSOzrG/twJq9Fb0uBm0fFCn234bXWZFs3efG\nLszGh5sO4s3V+w31fLG7tAFvrt6Pg9XNuGt6uoMRekdCOn5RuK+8EetyK51diE7RVTWzzlrHr57j\n3Ms7wk55Ny0PgP6L+YVZpZiwYh/+M+/Lmnu9o6sGpSTUs9/5vXIkGSbrLnhkdhY6E9SQ9Cm04ux0\nr63Yhy0avWVE9az92l7Yuy2wl7uxUyc1vbNt7wzjpcW7bVmmke1oaZsrWO6EwxLXvrYOt0zehIc/\n3d79fqLft6U9hL/O2Ko9gQPcqlkPhSX2lTdoJhR+zzPye/Qj/t76PAPfa7I/GMVYarMeb34a0zrV\n7a0KrDw42dDagfc35iOrsO8zL0Y5XVzEzv+pBdmRPwyWD7O7Bmhq7wzjhtd7P4zvadeNHi07iBdy\niTBZd0Ftc/IO/pN54Yvd2GZDwaSX3YeBoeNZ7/OlMdMlui2ttwmSU+y66ne7YJyZUdj9987ieuyv\niCRiM3U+mDhhxV7NaVVprmLWzW9vxA9eXo1/zzbXrZsV1U2RQYtUPGEFoOWGLmb337jfU/B3tIuV\nByy1phm7MAePzNqBn722FjUWe4nR36WyiZk7UMbFu0usFdvuUv2jMPfUqVgzWj0CfAgBYLLuCqOF\nehB3OjcSs0QJsRebtOcq+zWB+evH27r/1ur7NtH+OjPDeG8jVvd/M7tacW0LWjv012IWVDd3dyv3\n/kb93bXZoa6lA5c8vwLXTFjXfUs9lpcXQq5eQHg6GIzPrzY9ZqVr0GhXimEJfJpeGH8incvwilZo\ndj4U+vTCbEPTh8ISN721ASOfXIKlu8psi8MJCv+0jmCyroB4J7cgnQicWpN+WsM4O7S8uMtyaGHu\nroPedvfGo7Ljt3f6hPvJlgKMem45Ln1hhe7veNlTx8SV+9DYFrlTNEZHL0Bu1767emHq0LLGLd2D\n4Y8uwoqccs3F+v3OkBm2PlSr48fT0ytWooQ0r6oJb63ej8KaZs1p6EuzMouQlluF+tZO3DltiyPL\nqG1ux93vp+OeDzJQ12K91UFU0JN3JusuUKFMV7mGIZbeE4IX29X0duzzxcTRq9i8QSuiRKF60XWY\n0UX+7dPtkBIoq3fmIWS71be436Qrr6o5bo8w8fZTq31je21HUR3GLd2Llo4Qbp+6WXM6rd0s2bEb\nr3zz9xYzSU/Nus5ZpWk8iDtnazGeXpiNez7I1B+XS+ws4+0qZg9UNhpftsEz8VMLsrEwqxQLtpfg\nhS/MD6AUu/n8Xu4kw2TdDT6sgvFyv3dkc3mwPlYWab1HFPvZ3Y2cZrKTmqmLbkZ/B7sumKamHdA1\nnasnTQfKivt1PBAtE1Stm1l9FY7vntyoLNBVxumMI/osjZZtBbX6gnKRnVvYrf3n3bQ8XDVuNeZs\nNT+YWs9mSx9v1m7C1JOeC4L/e2Ix3l6z33RcqmOy7gL/per2iz3YQnYkfjaeLM3S/XCSxeVMXJmL\n2uZ2T0/qWj+Z0yEZTd5VaULm1G9lNBm2K/GanVmsc3m2LM4WZtZd77MLbuxlsRdapXWtaGi1r+mA\nlkyN5NbteieFdiVTsQio38+8Xk1tnXh87k7klDbgvo/s6eHLzooZKXv0tBNATNY9NGH5XizLLotb\nw2B3oehlbWW8mr3Rr6yyPHBTn7kmWEW71t/IfHrGZ0fB7FRBpDe0udv0JWs9JdqP86qabekpycgy\ngyCk0Fk+bmVozLv7ys31SGE6gC47i+tw+Uur8Os3Nxh6eFjfYrX7WTfz6+i5qFi5uxyjnluOC8cu\nQ2mdswO9NbrQe5aVB0xjve7SEPdGuXWkulHmNbV526NaH+oUg65gsu4CrQPpxcV7cMe7W5Bbbryd\nmFEKnd8BALkVTZpP8ZsdFCkRL9Zf9vq7dwDJYo8Xb7J+9p0W7YEhlpWa29ieVCoa2tDa3vsizujs\nVcnVnbhArmvpMDwqqF3NYJKtTfSh13DMNfgPXl6t+Z3m9k6syClHc7v9icBvp2zGgcomrN9fhYkG\nkjm9m0vrDk7SNutxvqZnT7ntnc0IhSWa2kN4Yp53Q9vbJfb4iHe86L2LVFTbYktMblKxGUzC+ThQ\nsHaE7C8j//xBBjIP1tg+X68d4nUAqSDZPv7RpoJe/5aQ3iccNh9D8U5QFQ3WHujrc7Ls+qeKbZ7t\nKEy9eFjTKqMh3/N+BjZZvCjx43bSQ8pIt2rNBgfKcaP98cOfbsfH6QW4+/un45RjB+n+3q2TN2FL\nfg2+943jMP3OC4wvOMFP3bM/6ox8m0/eCe5+xtva7Z1hzNlahMED7Dnlljhcs+4GXTXrzofhaEVO\nopJIxU4EvNDaEcLAQ/sb/p7WeX7+9hLM316CvGd/bDU0pbBm3QVBTR700u41QWN6k4MiWSnZ95U3\n4J73M5BjcBAJ3d0emgnKJxJXxhjb9+Ml6l5tO6cOW7Mn6aqmduwsrrc5Gv20NkddcwdmbCmAlMBr\nK3J1DxvfEQpjS1cSvXZfpe0PMDvNyO4xY0sB/vbpdvzp/Qys3hu/5xItXiR1Wku085DQs1Yq5bPm\nfwd7uxhWLp2wGE8QLjzdwGRdQU4UUAqVed2cqgGPnywk/s5vp2zGgqwSU/N2gpu/l9V1Wphguyl3\nYnFRvO0aCkvc+Mb6mOl0XvB5fBBrLT62z/l4TRc25/Wt2Y6drMXmduU9ObEfGnlw8NEeo9yu3lOh\n6zsA8PicHRjxnyWYsdndgbfspLe5UNxy24WSMJXLKCOcenBfz1zjNh1TMalxEJN1FyTbGVVstmE3\nJ0bYtrPocKLNY6IHTJPuEz4pido6Q5iz1fiDp6lqztYipMc0yZitc/sZSyq8y0DiVZA/NmdHn/di\nyz2jzXvs8qP/rsGry/Zqfh7vWIz03KhVY2rfsVtc14q6lg48/FmWbfM0xUJGq7U9dBVxLhSDnhW1\n/ijiE6pv7bC8Hnp2LZ+cDh3FNusuSLYzxl6xJujC1zVuXEBo32rVOSiSkQdMPSgZEz1gmky8RKCh\ntRNtcQam8ZJWrxEldS34YkcpCmusXwT55cIlVryo7dgeXtF7wW22n/XIQ6YDTH031t4y/c3Zskvq\nkV1Sj5+NOAlfO/YInb2U9J2ovKEV2wrqcOFpQ42EGpmfw9Pbxe3zUpAHuvF7Jd28bcV48ONtaLfY\nq5tZ/t56xjFZd4Eq/T57RbOA1yiI9bdZt7ctoB59utk0+8Vkk8fZNj0fmFOF1mr9ftoW7Cjypn11\nUW0LZmUW4rrvnOzJ8oNK7y6sezoHm8H84b10w985WN0cN1nX6ko39r0fj1+LioY2XP+dkwwv268X\npHpoN4Pp/e/XV+biwdHDepXrKm0WM6EIIQz3s27mgsiNi6jYZfzlQ3tGhdWTH8VbPzfGG1AJm8Eo\nQIUr7E/TCzFm7k4UO9IcRMRNrO1ea++3Ym+9msEY/K4TJ6lbp2zS6G/a/oXZmaibie6BGdtsW76d\nrJxTjXy3oLrZwpKsMfugqNka7XgOVCYe0dIIvXcAo71bzcw0P7pjKok9701Ysa/Pc0OqlelGJdpf\ntT4Kwgi4TqtoaMOlL6z0OgxXMVl3Q5Kz7Mdbevc37kQtS6JZ7iyuw0OfbMPUtDzc95E9V8tWY7I2\n3zhtTD0ozKwsUs/Drkat3lOBiSv22T5fis/N2tLYWqanF2bbPnBO3GYwOvvGjnuxHqcG2y+cGLhO\nJXbvux2hMF5fmYuXF+/u7lM/3iKmb8iPicO+GG5/Z5Pl7oKDrK0zhPyq3hf5szILceXLqzB57QHH\nlmvmOBq7MLgjlWphsu4CM2W6HU1nQmGpq9Dt2ZtHtNcGN06cWu0R9ax5U1sndsd0s5iwT1v9YSVk\n5CS29WANrhq3Gn/9eKsy3dKl5VZ5HYJhXiVxfmm+VlbfigvHLuvz/vOLcmxdjtnmLbbG4OS8u9aw\nby8l5sspx3h0UJjthnjG5gI8tygH45fvw8QVkQGq4q1BZ8wgOXbedV6xuwJj5sYbTMq5bZloe2mt\nmxcXgZ2hMEa/srpPJdEDM7Zhb3kjnpy/Cy0ePQAeTypedDFZd8HGA8YGebGj6Nhb3oCLn1uOH7y8\nKmFb57rmDry2wpuhms2uZ1tnCJe/tBJzt/XuRcOL01eic2Z9aydyShswM6MI87YHs8cUlWtDxy/b\n2z2qpt1a2pM/VGX3ptE68Y+ZuxNNcU6kTnaFmIjehwLNJGKeHOMa73tZs67wYYc5W/s2Axq3dE/3\n3xMS3N3rCMdeKNkXF2DtjqXZWLy+5G9s68QLX+TgjqmbUaWRC8zfXtKnVj1WkwOjDJulUixuYbKu\nKD0nggOVTXjok234KM4w8DuK6lFc14rciqau2oT4Jc0zn7twO8ngAzbJVn5mRhHK6rUvQFRMIGPj\nTfWBsgwx+Xu+vGQPXl2u3SWfFZNWe3OBG0+xRnMXr44Dvcvt0wzGhhR0zd4K3DF1c5/39Rxv0Xj6\nPEQeJyzbn7dRrMyyEs59H23Vt4w4K90Z07OIO03JvCmL3bpz9+ryvXhtRS6W5ZTjnKeWxn1uqV7H\nw5oZ+bVOhGf4onfRjhJkHkweiyp3s+3CZF1BesunP0zbgk/TC/GPmVnILtF+oC/Rju1lswizJ+cm\nE7Wlqp0MvZJqm2HSqv22zk9KiRW7yzF/u/3PFJil2mWfk93tJUvebpm8Cctyyi0uQ980diZbKnQy\nkEzmwRq8udr48RRJxnpvq9aOEMrjNGXoiE3WDS/NDPW3vRWxZeCn6YUaUyb2r1nO9PVvtOLqrukZ\nuqYLWrefTNZ9bG95Y/ffq+KMiheVaMAfzQErzIcVl9YIZNkl9Ro9lKgpYMc/gOTrtClBMy4/JBl2\nSs+vwe3v9K25jSfedrVyQ8XoV10ZK8FCrXPsdH/7ZDsW7Sg19B0nxG63eNuxrqUDH8a5oxlk101M\nM/W9ePvI955bjl+9uaHP+33arKdW8eIKP51ve+oMhbF4Z+LyoadQwHYe9rOuIhP7WLKryD1ljQk/\nd5JWkvHm6v14c/V+nPaVQb3e31WcuNu/fqYyHmcOXL8lq2ZuK6/eU4GSOu8G81FpG/9zpscjScaR\n7HD4wOWkMu5FStzpek+4q6Qed01PR8ajV2LooMPiznvD/t53AvWWBOl5yZ8b0trL4q3PZxnmaift\n4nge4vD8Kxvb477fEY6tWXcmkFBYon+/6N7j7L0pozXHqdZCUu/qTll3AGMX6n9oPqzW+IGWsWY9\nIJIV3rFDnEep0OPF/orefSIvzS5LOL32IBPG+7T1ipdbvU9vF0mmj3Tpqa8datAZOZEWVDfj4y0F\nqGv2ZvCO6M/s5YV6Ilr73Z4Eo4/eMnmTrnnEivcArhZPunlVrHyKZbW80nvcxNasO9Hs+PWVufj2\nmC969Jbk3APRTrC6r6i+r2kxkqgDwatZZ7KuIFUKBTsZvSjYdKAaP38jDa/F6Tkg2ZwCdox6LmGP\nKi5sa5V+TyP78U1vbcDfP92OBz6250LHaI2bQpvNF6IXsSr1OKMlq6gO7Z3uVR1a2SZG9ts+bdYd\n+DGeW5SDpvYQJq7M7e7z3SneV4Wpz6k7CSE+YBqfEOJkIcQUIUSxEKJNCJEnhBgnhDjGxLzOFkJM\nE0IUdM2rXAixSghxq13xBk2yJ5+/vOWnj51P4Zs5GH8xaT0259XghS92I6e0d7MYlXpSUSmR1CNe\nuH5bB7dY3c3qWyOJwPIeDzxa2Xe1LhQSji/g8I9rZe52hOZGSWA1zutHnmRPIDGmrDvgyHy91Pc0\n5uz+a+SC579L7etZqrGtE4sMtL/uKdEWcXMgNrvoqQDJKdW+26aFvcHEIYQ4HUA6gNsBbALwCoD9\nAO4DsF4IcayBed0GIBPAtQDWAHgJwKeIlMtX2xGv6kwNN5zk80M0knU/1OLnlMQMfmTiDK3+WpIW\nVX67DfursDtBE41k1udWYd429/rbl9KZZgS2SRKbV6HH61LSykN5RxzW3/Ay9Xj28xwsz0ncZNAu\nlpvBmPjO+GV78eGmAotLTuyP76XrXsbElea6a4237q8uM5/4W03I/XDOt0PQeoOx6wHTiQCOB3Cv\nlPLV6JtCiJcBPADgaQB3JZuJEOJCAG8D2AHgKillacznh9oUr/KM1sAl2y8P7d8PbS7eNo1lZ2V4\n0mYwDhZGdh3/ng6oYmP/1sEqDrXtKWuI23uFXruK6/HrtxJ/v7KxDZ9sKcT5px6Dc74+VPe8MxJ0\nzaryCcuO49TOtZNx/or6r6XkSs+yza3J76ZuQd6zPzb13US8TuiyS+rx8pI9ySe0yOiAhXaZlKAL\nzIJq8w/z+3Hgpui5MByW2F5UZ9t82WY9hhDiNACjAeQBeC3m48cBNAG4RQgxCMk9D6A/gJtjE3UA\nkFJ686SWy6qa4j8pn8jussQ9qBzSX52mI0bFJrZaFzLBOjQpSoVbu/+Zt8vS9x+ZnbwXmX98loXn\nFuXghtfXo8ZEGdCXdDxZj991Y983zVycqlJiSRl5KJGcJ6VEU1snfvTfNV6HoqxEh7Seo12B4jSu\n/8zfhWtfW2ffDBVdT7PsaAZzedfrYillr6pbKWUDgHUAjgBwYaKZCCFOBnAxgC0AdgohLhNCPCSE\neFAIcYUQImUehn1i3k7D31mYlbj92yH99G++lvaQrfu5sLnPmWQn/rgJhIUSqrwh/giRfhR/27gf\nh99YvROi5+s9e0FaEqdHJDO1nU7/ti3tnaYGygGSx6YndDvLlb99sh0rcsrx9WN71yvx8LBO7/Ej\nEWn+EhRu30FVoWIjkZYEvTJNTcuzdVlqbwnj7GgGc2bXq9Y9q72I1LwPA7AswXzO6zH9cgDfj/k8\nSwhxvZSyb/cgMYQQ6TZxScIAACAASURBVBofnZXsuypYs7cS3z/zeFvn2d/Apc63xnyBS844ztbl\nWxFbk27mQTsrbng9DSsfuszwQ7puC4cl+iWJMWgFmF8ka9YWe5K14+GoSJt1+37xcFjie88t7/Xe\n7K362uDHe5BPtX2xsrENt0/djLsuPb1XV7fW2wgn9vtpW/Dtk46ytAyn7Uwy9oWdeg7253du586q\nHVOxfvnm+j7vObWNFL9uMcyO2upoKaPV2Cj6/tFJ5hPNTn8BYDiA67vm/Q0A7wE4G8ACIUT80TLI\nNqGwxIrd2iOiek2zn/UE37Fy3BZUtyCrqA5NbZ246W3z7Zad9tMJaxEOS+RWJD7Z+a0MszPeioY2\nJWuf1u6r7PVvOx4MlTbNJ2p3WQMKa5K3p423efeWN2L+9uKY6dT7HQCgOMGIz05YsqsMH2129kFK\no2J/mmRjXySj996qoruEUhI2g9H1fERfnycZOdgu2wvta5OejNfPXdjNjaYl0aM02Zbr3+P1Tinl\nLCllvZQyF8BvEWkeMwzADckWKKU8J97/AIz1qu8htetwjRHC3mHWE81q0Y4SPDpnR5/3rZ4EQmGJ\nV5bs6ZOsXDNhLSavVaMLtZ3F9Ri7MBs/eHmVoe9Z2TZ+Orn+Z94unPf0UtzrwQBPyXb/F77Y3evf\n8R6OMrOtVXrA9M8fZHodgilubMEily8Q4ukMhbF6TwWqbXlewhwppbIXcWZYPfcZpSdBza9q6rON\n03KrNKZ2nlNJdYB2IwD2JOvRSyWt+3hHxkynJXrfsQ3Awp4fyMieNafrn+cbDdCP0nIrk09kgNJd\nuBnUT6P0K6lrxV3TM7Bmr73bDoicRFbv7Xu3YU9ZI56cvwsVDW2W5m+la7ie3l57IHCFlF3rE+2X\net62YtQ2609I9lc0OrJP9RS7jvY0g5GQCg+5rbWGXldUxMYVtONJyxPzduHWKZsw8sklfUaVtsLO\n3zPZXcNUkCi51bOvfripAA99st3GiNSkUkWFHexI1qNVQsM0Pj+j6zVZP0zR+TTEPqjaJZrMH24g\nNt9aml2efCIDvNxvba9Z0JjfJge74Uq2+crqjT2EGnvb0bXa+Tg1V0G7XahHtA21noukO6dtsbw8\no8eAXScalU9YqoYWe3x8nlVicX6Wvp6UXYO/vLchv/vvpxdm2zJPIFJ2GnnANNHa3Pmu9WPR7+zY\nnz7LKERdSwdmZhSipM77uzpOULV8McuOB0xXdL2OFkL065loCyGGABgFoAVAssa+2wFUAjhOCPE/\nUsrYRnLf6nrNsx5yKvJ2z7U0cmNs140m5rFqTwW2F2r3R51MsgPf6OrFdgX31hpzPWoY5cfyy4mL\niWh+866OHgjsrGXUK95Q2Wa2gsrJul/8Y2bybje9FJIS/Ty/H2GTJLvrgUr3j0U/MXK4P/TJNizZ\nVYavDT3CuYB0YBGlj+Wa9a425YsBnALgnpiPnwAwCMA0KWX3USaEOEsI0atnFillJ4BJXf98vmdX\njUKIswHcBqATkdFMKYXEPpxkNvG/8fW+T6LrFWlLafrrSpu4wnwf0n6tlY8msc98rv0Yy4oc++5u\nGe28dPGuMrR1WmsaZfcDpnbT2ncUDllJ8S7sEnG7TbjR0jpo5ay9HRcn60hB/8ZbsitSH3qwutli\nRGoK2n5k1wimdwNIAzBeCHEFgGwAFwC4DJHmL4/ETB+9xxa7F48FcAWAWwGcLYRYCeAriDxUOhDA\ng3q6bqS+/Lzj2lGzDgDtIfMNeJ3efLXN3o33tcDibX7HObDxo8m6ENrHxuJdZdjnUTdymw5UY/yy\nvfjbD833Niuluj2uAFA2K7c/LGdX1Giy7gUjZbb6a6Of+/2su7s8uzgxlolfK5K02NIbTFft+rkA\npiKSpD8I4HQA4wF8V0qp61FjKWUzIsn6E4gMpHQPgGsQuRC4Wkr5sh3xpiI/3w7vCIXx0aaD+Hhz\nATpDYRgY36mPQ02O5Cpl4pOI3bUnTrJrT7hg7FK8tsKf187RwyHZr3bbO5vsWWCygbzi/Cqvxdzx\nMJp4q1+zHp+RI8lK8zpNCm+zeIwOq67yqUBPaH5qY51XaX+tdaJyQOGfVpME8Nhs4wNBJp2vHzdG\nAnbVrENKWQDgdp3TapawXQn7mK7/ySZe7rdWE9k5W4uxvKtJwqa8apRb6Hnl+CEDTXWTFpSrdDsL\nsLL6NkzfcNC+GTpkfW4Vvnv6sb3e03vxqqdfcZV5cZHuZm2+0ncOXGLXA6aqSPabvrBod8LPVfLf\nZXtxsc0DDAbr145YtNP+ft6Dtp3c6GedFKBV/rlxrhPCWvddy3u0Hf40vRCr93gwYJPND5h6xY8X\nHVYj/vVbfZ9tj+Y3jtTMxuHV7qHyHTVVQ7O7WZjT62m4zbpDcSSi9ziTUibtJtVKc0Yv2N2zWyJ+\nvHhts6nb4lh+3BaJMFlPEUHbcc0ym5tJBGMbetUGWzXRJFblZBYAynt0CWo0UgFvEmK9y/TjhaOK\njDaDUVlTe/LEza0LbFXF+7llzKufXP6SsUH89JIAWtpDqPFwkC87MVlPEQEqzz3RHgoj14Mu/OzW\n2hH23b7gRLzRCy+3toXZ/OL619NMP0AohEfNYPROZ0NoqZ64AUDYXxXNlvEXtzYoUqoor2/Dhc8s\nwwXPLLN9kEkvMFlPEQ1tnd4G4EIJ6+R5+5MtBUmXHYSa91ShWjNfrV2nsKYFD3+23dCIq1EC6q2n\n3faVN/YZs0A1ThcLnQazdbfLqcZ2e889frs+cyPe7kUE/Hg3Yszcnahr6UB7Zxg3vbXR63AsY7JO\nrlCltxSzBefCrOQPwCzaYf9DMuRMc4nXVuxDgYv9C1vZ/z9NL8Qjs3aY+q43D5hqf7aruP7L6Wxa\n3nOLcpBTWp98woCK3d4Hq5qxZFcZOuK07a5r7nA9n5u0ar+ph/q1+K1OxO4zX6L1Z9OyL+0ua/A6\nBFsxWafA8PJyQEAEdnCJIJqztRgXP78i+YQu2VmcONlckFViOEnpJ4Qnd3sSJQxXj1+DtV0PENoZ\n246i1E3We6pr7sDocavw+2lbMH7Z3l6fTVufh5FPLcGv3kw2mLja5m4r9joEZfntQob0Y7JOgaGn\n/aoqNfxe8lvtSxBOQF7cuo+0WXd/ucncOiVyS9rO31XlB4XdPN6mpuWhtSNSo/7q8t5jIDw2ZydC\nYYn0/BrX4iH7j/3EI5hSUDFZJ8cJIZRpZ+jUaH+qrB9RT6o1gwGcuYCQUqbsMyM9V9to+3Vynt0V\nRAmbwaToMZAKmKynqFQ9qO1sO9kTc3W1eT1wjB0Xc8aPWeFJTyFebGkV7yC4pbUzlLLluR+4UZHz\nSXohANasBxmT9RT10wlrUd7QmnxC0mWvj/ov99t53Y5wv//iShvm4i0z/RF70uRJ5w6mNVm0OZuR\nBFRKdfdrp+Ma/cpqXDsxDZ0+GyyIzNE6pkvqWpQ9Bsg6JuspakdRPe6enuHKsioa2gJf83z3+xnY\nVljrdRikweuHf7NLrPdM0Gi4+1V/nrnN1BIHaWAgM7YV1HbXrpJa3Dr31bV0+O55JDfVt3Z4HYIl\nTNZT2BaXHjTKLnGnpwavLwj0dO9IxgXhFn+1R6PoeTKCqe7pEk9pZMAjKZmm5FexNyol2dwOJuEx\nneoHQQLPL8rxOgRLmKwTEZHr7GwG4/UzCSqwclHLHmIcZPMVcwDqLjwxfcNBr0OwhMk6uYK9pZBZ\n63OrvA6BDNCbTNiZc0ioewfGrahK61tNJ3J//sCdJpGpaHxMF5pOUvMIIDswWU9xbp3f3Kj44gVB\nMP3t0+2eNSPxu9pm/7bTNNIMJhRmM5g5W+MPFtTU1om2zlDC77Z2JP6c1JFoT1f0epVscIjXAVBq\ncKO/50h/tiytgmjdvkr89P/+n9dh+IqUwBPzdrq+XD2JX3p+DY454tCE0xjtDYb6Vlj89eOtmJlR\nhCMHJj7Vc/P5R8J+1vlLBhZr1skVPJkSuc+LLkXfXnsg6TQ3vJ6Gkjr7uo4NS6lsGeNlXDMzigAA\n9a2JexJSdduRfgKCv2OAMVknV/ABMHX4sUD/JL0QNWwKEyhvrdkf9/3iroHLjDSDYfFijart/ckY\n/orBxWQ9xbnVztuVfpDZZj2wVu+pwAMfb/U6DF/ZU269b3cv3D8j8jsbagbT9R+Zwy0XDLzoSuyg\nj7s3ZbKe4tyq8WauTlat3F3hdQi+UlDd4nUIrslQuOtBX1xE+CBEitA6l2aX1KMjxB8ykacW7PI6\nBNP4gGmK63ApWXfjAdO2Tg63TRQURprBLM0u92XzLlVw0/mH1sXf/TO24vxThrocjb90hPybI7Bm\nPcV1urTzhtioVBm+qOmjlCalxIHKJq/D8B2zFRZsPhEMm/KqvQ6BHMJkPcV1unTbjLk6Eek1d1v8\nPsN9ycWyb9r6PFPfY/HsH7yuSk1M1lOcW8c9a26IqKdEjVzu+4gPE5vR2mG2Zt3mQIjIVkzWU1xj\nW+L+d+3CZjDq4E9BQcSE0zw2jfMP/lLmGXkORjVM1skVTBDVsXRXmdchEKUMFn1kJ96lNs/P247J\nOrmitpkD2qgi2o81kZdW2NwVJ2uHzfNxDpNS3LoTTuphsk6uWJZT7nUIRBRQZxw/2OsQfI25uj+c\n99RSpOVWeR0GeYDJOhER+dqxgw9j7bAV3Ha+0NIRwgtf7PY6DN9im3UiIiLqww/tZNmEiEhtTNaJ\niMj3mG6ax2HqidTGZJ2IiMgh87eXeB0CEfkck3UiIvI1KdVtbtLJfmuJyCIm60RE5Gub86rZDIaI\nAovJOhER+VpYAkt2crAvIgomJutEROR7D36yzesQiIgcwWSdiIiIiEhRTNaJiIiIKND8OySSjcm6\nEOJkIcQUIUSxEKJNCJEnhBgnhDjGwDxWCiFkgv8H2hUvEREREZHqDrFjJkKI0wGkATgewBwAOQDO\nB3AfgKuEEKOklFUGZvmExvudlgIlIiIiIvIRW5J1ABMRSdTvlVK+Gn1TCPEygAcAPA3gLr0zk1KO\nsSkuIiIiIkpxfu7e1XIzGCHEaQBGA8gD8FrMx48DaAJwixBikNVlERERERGlEjtq1i/vel0spQz3\n/EBK2SCEWIdIMn8hgGV6ZiiE+CWAUwG0A8gGsFxK2WZDrERERESUYvz8gKkdyfqZXa97ND7fi0iy\nPgw6k3UAH8X8u1wIcY+U8lM9XxZCpGt8dJbO5RMRERERec6O3mCO6nqt0/g8+v7ROuY1B8BPAZwM\n4HBEkutnur47QwjxIwtxEhERERH5il0PmCYSvfOQtG2/lPKVmLd2A/iXEKIYwKsAxgL4XMd8zokb\nSKTGfWSy7xMRERERqcCOmvVozflRGp8fGTOdGW8j0m3jCCHEEAvzISIiIqIUI3zcaN2OZH131+sw\njc/P6HrVatOelJSyFUBD1z/ZqwwRERER6SZ93HejHcn6iq7X0UKIXvPrqgUfBaAFwAazCxBCnAng\nGEQS9kqz8yEiIiIi8hPLybqUMhfAYgCnALgn5uMnEKkJnyalbIq+KYQ4SwjRq2cWIcRpQoiTYucv\nhDgOwDtd//xISslRTImIiIgoJdj1gOndANIAjBdCXIFI3+gXALgMkeYvj8RMn9312rMF0SUA3hZC\nrAKQC6AawNcAXI1Ie/gtAP5uU7xERERERMqzJVmXUuYKIc4F8B8AVyGSYJcAGA/gCSlltY7ZpAOY\nDuAcACMQeTC1AUAWgI8BTJJSttsRLxERERGlDj8/YGpb141SygIAt+ucts8mk1JmAbjNrniIiIiI\niPzOjgdMiYiIiIjIAUzWiYiIiIgUxWSdiIiIiEhRTNaJiIiIiBTFZJ2IiIiISFFM1omIiIiIFMVk\nnYiIiIhIUUzWiYiIiCjg/DsqEpN1IiIiIgo46XUApjFZJyIiIiJSFJN1IiIiIiJFMVknIiIiooBj\nm3VK4N8/Hu51CERERETkQ0zWXXD9yJO9DoGIiIiIfIjJugv6C//eeiEiIiLyu9rmdq9DMI3JuhuY\nqxMRERF5Zkt+jdchmMZknYiIiIhIUUzWXcBWMERERERkBpN1FzBXJyIiIiIzmKwTERERESmKyboL\nBNvBEBEREZEJTNaJiIiIiBTFZN0FrFcnIiIiIjOYrBMRERERKYrJugvYZJ2IiIiIzGCyTkRERESk\nKCbrLhBstU5EREREJjBZJyIiIiJSFJN1F7DNOhERERGZwWSdiIiIiEhRTNaJiIiIiBTFZN0FbAZD\nRERERGYwWSciIiKiQBt0WH+vQzCNyboL2HVjMNww8mSvQyAiIiITpNcBWMBknUinkV8/2usQiIiI\nKMUwWXcB26wTERERkRlM1iklDB10mOV5SD/fQyMiIiJfYrLuAlase++Nm8/xOgQiIiLyiJ8r3GxL\n1oUQJwshpgghioUQbUKIPCHEOCHEMRbmeYkQIiSE+P/t3XmYHFW9PvD327Nm9kwyk8nMZDKZPckk\nk2SSyWRPJiEECCErIRAg7EtkXwRxAQW9Ksqu4oIIXBeuC/70qhcvoqCICi6AF5EtsgoaBFnCkuT8\n/uiepKe7qruWU1v3+3meoUl3ddXpOnWqvnXqLEpELteVViIn2JyJiIgomlSEu5hqCdZFpB3AgwCO\nA/BbAFcBeArAWQB+LSJjHKyzEsDXALylI41BEkZ5OSHKd+VEtN+Fq7qDTgIRkWW6atY/B6AewJlK\nqbVKqYuUUkOIB+3dAK5wsM5rAFQD+ISmNFIeU4y0iSiBw+kSUZS4DtZFpA3ASgA7ANyQ8vFHALwJ\n4GgRKbexzsMQr6U/E8ALbtMYNF4WiIiIiIIT5To7HTXrQ4nXO5VSe5M/UEq9DuBXAMoADFpZmYjU\nA/gSgDuUUrdpSB8REdE+bJlIRFGiI1gfbvz3V5PPH0+8dllc3xcRT9epThMkIg8a/QHocbpON3hh\nCJ4CcOy8iUEngwx8csO0oJNAeYanZCKKEh3BenXi9TWTz4ffzzr9o4gcD+AwAKcrpV7SkLZQYAfT\ncLhwVQ8+tXF60MmghG+dPIgvbJ2FDbOag04K5ZkIPw0nojxU6MM2hiPVjOdHEWkFcDWA/1JK3e5m\ng0opw0G1E7Xrs9ysm6KrvKQQh8+egPd/56FIt13LFXPbbA8SRaRFcQGnGCHKN1G+7Os4Yw3XnFeb\nfF6VspyZmwDsAnC6hjQRjZAcnDt9zhHlgk5E+5UUMVgnyjsRvojrOGM9lng1a5PemXg1a9M+bBbi\nwz/+IzEJkhIRBeCric8vSbx3h7vkEhFRPouxaSIRRYiOZjB3J15XikgseUSYxMRGCxCvMb8/y3pu\nQXzUmFSdABYD+CPiEy/9wXWKKa+JSLTHcCIiV1j8iShKXAfrSqknReROxMda3w7guqSPLwNQDuBG\npdSbw2+KSE/iu39JWs+ZRusXkW2IB+v/rZT6oNv0EhFRfovytONElH90dTA9HcB9AK4VkeUAHgUw\nF8AyxJu/XJKy/KOJVz6LJF8kX5x50BEREeWXKN+ka+llo5R6EsBsADcjHqSfB6AdwLUA5imldurY\nDhERERFRPtE2dKNS6lkAx1lc1nLlplLqZsRvAoiIiFwpLy4IOglEFIAo91Xh+FV5bvL4quwL5QId\nhTTKJZ2IAMRPBSzKFBWLu+qCTgKFAIP1PPfh1VOCTgIRka8Yq1NUsI8VAQzWKQ/lyxDLHfUVI/59\nw5GcvJf805ly/IUFa9UpSvLleuWHKBd9bW3WKZry5UQQ5ULqxClL2nDxQZPxxju78eV7n8LosmIc\n1NsQdLJMnb+yC1femW3eNIqKH56xEJd87+Ggk0FElBNYs055QUb8f7jvUD62ttf1Os4Yik8cXFFS\niLNXdOHY+a2IxcL7u5d21wedBNIozJUAImD1OhFFCoN1ynnjqkowMKk26GRYpqMGvKIkWg/NGDuR\nX3isOVMY4pv9XMa9rs/1W2YGnQTHGKznuVw/EVy2Ziq+d/oCFBbwUA+zKE1WsWVgQtBJCD2BhDpH\nw5i2/omjg05CRqVFHPIyCBKCx1SVpdGq/DET5Se4jGAop62d0YTGmlGBbJs1UdYZ1XZuX9buf0Is\nYM0seeEbJw0GnYSMQhAz5iXudgIYrOe9MNy1e8rg5727Z68vm97jMKrL8RwxZLSnCkJ6bLpJ1vjq\nUn0JCbGQZh2A8D7FKS4M9+XYzywt0zBx1cUH9WhICQEI56MoB8J8Xsom3GcH0qJtbHnQSQhMkJXb\nuVYD6+WJThntrJCeWXMtX70Q0qzbh3loX5R22Sc3TMOJi9qCToYWYS9L5A8G63nglhMGgk5CYGIa\nz3R+Xaxy/mmHAaN9m397YaSl3eGduXBWS03WZY5fMMmHlOS2ooIQlYIIRevLuutRwGaIlCLKl1YG\n63nOyfksSo/ygyqc1x8Z3V7nQTCqWT9qsCWAlITHFeumBZ0EU9luKAWCNX2NoQzYlTJ5khNCNx83\ngHUzm4JORuTkUoVHUJ0iB9v2j6B2w1G5MaFe2IdtzoTBeohUBjDcnpOT2qVrpnqQEm/orFm3Y/X0\nxkC2G1VG+VRfGZ2bQi+MrSgObNufWD8NXz5mtunnVm7yYzHBqUtzoylCUBZ0jB0RNAXJz9sbt2ft\nHIrVsWUgmEqLg3rH4wtb+3HTttlY1Dk2kDToFuXjgsF6iPzhwwf4vk0nB29teXBBhF1BFs4Oh9Ot\nR/h84tj05uzNKsIiIpWyjq2b2YQtAy1YMWWc6TLZaqjC2omT9ps8viroJHgml86hQTXnicUEq3ob\nMNQzLmeeVET5VzBYDxEdY4HPbx+T9l6mghZUzbNfrP6+qY36L1w3HJkbjw79UBCTnKm90SWoR7Yf\nOXRK9oWyJE3nDc3HNTcH+uIxsyN/K2Glz0A2uXzmz/Xrmh+4B8OFwXqOOclmD/hcL5BWf9/RgxO1\nb7u7oRJfytCUwCtzIzRba7IoPbHJNYUxwQUHduM7p81DTVn2fLBarnTccMzV2AxkbEUxFkfsptBo\nH54x1Ol+vTl88s/l30bORfkJAYP1HGN3lrmw1EB0Omwykk3Qv2+Ug1n/3Cb56iNmuFuBC17lYy5Y\n0JH+1CsTPw/d+soSbF/Wgf6J1gLj5/61K+PnYW0qtLizDiIS2vRZtVfDDzBbRW+T8VNGLzrletXE\nI8pBWVjk4i6M8k9isB5h6y2OEpDpJBuWAtng0QgzYfl9fhpfHcyMrUD8aQIZ+8ym4G6idHv+1SzB\neuQbmuQ+s3PjuQd0+ZcGr9abh+d9yi7Kx4X/w4+QPhoOvLAcvF7VhPhRwzK9uRo1ZcW456//SPvM\nSdAS5eGl8oHdPL3l+AEs6hwLEXs5G+WjIOo112GnY/+anRr9rJWOP/lM/zFvvrvH1Xqt/oKZLTX4\nwzOvGn5WUhjDO7vdz3Y9raka5x/YjZLCGI744v2u1+eXXLwORfmJC2vW84DbDqbVo4p0JsdQXUWJ\n59twK9MFMrqngNyzfVm71vW5nZimuDC2rwyGNYb17mbZk9W6FnQ+lBXbbx6XzCj9dluUmJ3PvMqy\n7nGV+OhhI4f99er4sNr88QMHTzb9rLJUT11mZWkhlnTVYbDNXjM4omQM1nOM3ZNftpPaZzb1uUiN\nNeXFBbj44B7t6z15cfBjPKdeEI1G64mKqNSWFmkYVSlZag1TtjIzrqoEm2dPAAC01JZhTquzDpJ+\n1gKFeZIgrUnzYZe21JZ5vg2jNuu/fP+Q6/V2jTPvc+I2G0aXF6WVTbOy9In17kYAslp0ajJURF26\nZqqjSQNzRVhvtLMZ6glmEimvMViPuNRhzexe2LIVyIFJtZ5eyE9aNAn3f2A5xnpQs56p1iQoXzt+\nAP/vfQuwpi/DpEkRPUm6pesw2zCrGVWaasWA9GYvWw1GDtrY34xrt8zEmcs7ccvxc3H5ul7cdsJc\n/OCMhSM60eVL1trJyyAmf/LynHbbCXPxwUMyn3t0HwfnrOiyHVztNdgFx2WYcdbtLlMq/XebBcMb\nZjW72pbbJhz/sX4aDu4dj8JYOEKkphr/+yHpOkav3NSHLx7dr2lt2U3J0fkDwnEkkiMCQUmhuyzM\nViB13l2fuTx9uLHm0WWoLI3Xbpy+VF/zBT9ng7VzESsqiGF6c01gE124EZWaltKiAvz03CVoG1vu\nyfqNmoVduakPa/oace4BXehuqERRQQwLO8e6akIWkd1taPgGx+1vuHpz9Drltowpw4k2h9DNyGAn\npp5zlk+utx2g7t6T3h5bKW+f6KSeKosLY4bzKxQXxlBs4drWPc64Q7vbn3DEQAtiMQnNefor2/wf\nAliXjf3NWDm1IehkRB6D9YhLPflV2AxSs53URETbybsny0gh25d14MyhDi3bsptkr65PZnF8ps1F\nJSjWTefvHldViiXddfpWmCRf8ydZttl57dzAZlp2rcURr6xvTO/qnHJ/TnX/Q/YYVK0rKNNzkxcj\n/IgIvrptDgYM5oa4xMKT0c8cbtxM0+ruzbbcPA3NFnWcL3oa/K8tjup5LqrpzobBesSlPs41GyPX\nXPYjO3UbXj1BLi8pxLkru7Wsy+8aEbu75NyV9h9be+G0DE8zkscFt/po0c2hofu40jWaQVDNucNw\nfJjZ1J+5mYKOXcZJsswZHZN2j5fdRu1gHKzHKqOtKaVQWBBDX3N12mdWKp7Mmk/qml/Dbdt5L+nq\nAJtrQnzadIXBeoQZnY9EBJ87yvo099liWp0Hvp+FKOjJkIaZtY1tHl2G75423+fUpMs02+kZQ50Y\n6qlH34QaXH/kTB9T5U5Ist4z3zhpMOgkuG5+l8zX/ArJsRF8vbp5M5ggOH3SYDr8ZOLVbdPKcVXe\nzP8BZO7Mm01LbRlWTx+vMTXpdA/deIjH6d0nRy8ADNYjzujcaqdWOdtJUiR9VJUolAUv2l2aBd4K\nylGHtZktow3ba4blpmZUUQFu2jYH39++AG11Fi8sIWlm4CU3x5ad3WO2nQm1wU16NSzbOWa4PFjZ\nV2ZFJ8wj1Lh1WNJ0pgAAIABJREFU+bpey8sa7UHDmnWbaTCqWVfwdnxtszX3Txy97/+HZ322kvtm\n6xs+7t6noVmlV9e7IwdaHH/35MVt2q5x2Zq06TLewxufZBEITxxhsB4yGUcJMeLypG1l2dTOUlG4\nhoakX1DoZTrfp35mJd91t2tNHZc5CGE73GvKinHNEXo7Xtq98MeyBetuEuODoM9hq6fbPM+n0FHO\njNqsA+bnBC/32cop47B1sAUzJtTgW6fYeHJkktbhw7Os2H1TkbBeSnSl6zyzGWsdbsDtSD5kjMF6\nyJy4yHzoLCNGJ207F96sHUwhKC1yN4EH4H8bci+awTipyfDi+ja2ohhLPeo8mcxJDZubC7rRV7fO\nnehbzY8ZnTW8Oo5KAXDYDGsdL0+xONeA3d+YrXzp2GXK4P80rjRQBS5HGkndvyKwfXC9Z9AMBvZX\nY12mSeVEcPnaabhj+wJMb66JL27hIBplcm1KPldfdFBP4j2grc7+CFFdJiPO5IL57WO0T3oYdEVZ\nFJ78O8FgPWSsDFc1TGB8UUwtLLqPXTs97ac1VeOwGY34/vYFvhYiL04Yps1gdHeMzLCjrjliBr53\n+gJtE/9kCshTkxHESTAWEyzsSG8qlEmBxwlNXXuhz1cnOz/vtKXtuODAbnz0sKk4ZJr9NqPLTG4K\ns/9k94WiwcPH5l6MbOIno9Tbvbnum1BjsGLv9ouyudetLFtZWpT1hvS4Ba245ogZ+M5p8zExbcKq\nkftsWXcdbj9l3oj3wthfR0HPfacI8J7ZExaH6zS7kQ86iPbyfOIHBusRZ1TMJo6xPoOekxpopYDv\nnZ69c+QZQ534wRkLcc0RM9HblN7b30vZHtPrlLGGTPO177AZTZigcYbETNnf6aADlJtrva4c0/1U\nJdtPutHHCT8Ae0FZeUkhti/rwDHzWlFYkOnGzPizT23sw4kL05/2WX3K5CYnrjnCgyApR2rddDzt\nqS0vxpeOGTl+twJM95HRFsMwWsrFB0/Gj85cZPp5SWEBDpvRhFkto02XGfbV4wbShpHsqM/dmnWB\nGHY0dmN0wKM4mZ0fv37SXJ9ToheD9RzUUV+J05e2o62u3LOZw2a2jMb46vDeqfrZ7KbYRS23045i\nXv+6D62egpLCkY+YvW7nq2v1ksgOv2pyurPMH6Cbnd+VfOPi5CamrrLEsJOem2YwWyx0rPvJ2Ytc\n7dej5rZgXJX+WZHDzG72KgUcMGVc2ntW/fisRZbyMllqEnWV+bBMXmTEyw67bokA7+0x67vgLN3t\ndeVYP7MJxQUxfGj1FDfJc8Qo2X0TaqwPkhBSHKgzZOycLLsbKk2Xv3BVDy5cFW+r98Kru0zX4aQ8\nDn/H/lf9O2kdN7/Vt23Zabqki7bA1uT9yeOzB0pzJ9XiN0+/MuK9MDQv0F6znuUneTnjo1vJMYzO\n/ZItNsq0y5pHZx/Npm7E+Nn2071loAVbByfioGvuNfzc6Y1nSWEM7+zWUxPppnbcqxtnpZR5cJmy\nzckhmtbd6XnHSZEYX12KF1972/Ly0w3GkHdN4wGwe6/emvWpjdXYNHsCPrFhWlqFjx/CezZ2hzXr\nEVRbXoyB1locO7/VdXDkqNNk8PFYRtuXteOowYna12v2s4sLY473ie3Rf3Szkf1W2rB7cWzYDWq8\nPlmHODY3yKP9b2R6AGR7H2cL1pX5ctOaqrFloAVjK0pM2wOPrCm1nratgy34zKY+z5rd3bF9AU5Z\n0oYfnrHQk/UbMTpHGw4soGFbOgYTMGP73BCi68y3Th7EisnjcM0RMyx1yDyotwGN1aWY2VKDM4Y6\nfUihMyKC3WY16w7Wd8GB3ZjSGL+JSw3U/arUMNxM2IMWC1izHkG/u2TFvouZlWMwU41aiOMOU001\no/B8hqcFFxzYY3udbh5VFhXEsHvPHsPPst1MbehvxoN/+xe+9cCz+9NiISm68s3O70491oyOvTCc\nEv2eEMvvMmT0867ePANtdeUoKy7Ais/eY/g9vTXr5usaV1WCWS0GnReTfGL9NHx8XS9EBB/5/p/T\nPnd6YU9uvpVpFU6P08njq7TVKLfXVeDxl98AALTa6GcE6B1t58yhDlz7sycwtqIYa2c24Y/Pvmqy\nvPWNnrRoEr5079Om29TN61hsbtsYzG2Lz+j8+Z8/mXX53qZq3HDkLE/7Tun4zQIYzvUBOKuU2L7M\n/bj2ZIw16yFjpQAm1zpZqREbV1WCHpP2n26aweQbs59dXBgzvZBly56CmOCS1ZNtp0VbMxgXeRmG\nwNyI38en79tLHInfOGkQzaNH4aDeBqzpa8T05pqMNaOZAodMwbHx1PbGy1+5qQ/fPnU+CjNU4+9r\nRpdhm07bICfffHrdVrgoQ4ddK64/chZGFRVgVFEBPneU+75FZvsz24AD5xzQhe+cNh93nbcUpUUF\nrvfaJ9ZPw9krTMbutsHonOp2mMGhnvp9/292TdTF00Dd4nJjsnT2FAHGVJTg6yfOxQkGHcn9trG/\n2XRY4j6LzYnC3CzRDQbreUBE8M2TB/GFrf1YMbl+5Gc+jqUd9TJk9rNXpnTScivMHZJGMKpZD0EE\n73XNemr++J1fwz9vXvsY3HvhMnx+a/++wCDT/s84aFGGL1aUpj+ArSwxfii7sb9Zy2hFyWl1Okyp\n0WGwamqDwxSl+9Yp89BU43w22e6GSvzmkuW4/wPL9zUdsMpOx/Sbts3JOBiAiKB/4mgt4233Nceb\nOJUbHB8qQxqtuPWEAVy7xd0IQVsGWrCpvxnz28fgc0fNcrWuTMIy++4v3z+UcaSc4fyY3zEW56ZM\njhTE9XqgtRY3Hzdg+Nn332et6Zlh80w3iQoJBushY7cNutWla8qKsaq3AaNSZnRzUyCDuoMNy4lw\nqKceJy6chDV9jb4PWaivGYzZ+9m3YHSsbhmY4DJF7g0Hen4dnf7XrCdv2/rGnY4/X1QQw1FzR476\nsaSrDp0WJqsyOo6mNmavIUu+4Zo4pnzEdPSA+ayLyT8xdcsnLpy0b/QTHaeQWS2j8cv3LzMdvtDK\n7q4qLcoaJButZq+NH9BeV4HPHp4+463ZedTsmDIKwL2UmrxFnXWmTzOs7o7Cghg+vakPXz9p0NPR\nQUJyicKo4oKMN5Shq4XWkBy/m0H6RVuwLiLNInKTiLwgIu+IyA4RuVpEsg9uun8dF4jIjxLffUNE\n/i0iD4vIZ0WEc9gacHtS8PO49mNTi7u8n9lz2E3b5uCDq6d4csI7foE/jyTdpN3o2Et+zOyX1Ccb\nud9m3XyLmYK4TN/Ldhykjj0diwn++8xF2NRv/7Rca2Ec5tQ8/ObJI6egn9c+xvB7mX7F2Qd07fud\nVm40xlZkT6eImAZDXh4XhpMiZdignSJhtmxteTG2zW9FaVEMHzjYfr8gI0F3Oo1qWKeUjTzNdFwk\n/39q53QLe0d70xkN+ev1pHhB0RKsi0g7gAcBHAfgtwCuAvAUgLMA/FpEjM+s6U4B0AjgFwA+B+Ar\nAHYCOAfAn0UkfFOJBcxtLXNkmlxY0D2uEp/eON3Zl13uBrNssJI9qYGSCHDeyi5cvrbX85spnW3W\nF3aM9eTGJdsuTG1C4OU+EzFYf4iKUKbjTef09kC8r4aTibPS1m3wXmpaU5vCmP3M5OMvPfjYb/nk\neqzpa0RjhuYhy3vcNW8LXa2lRZlSfemaqXjk0gNx8uJ2ky/v/7bRHB+6ZzDdv2xIqrIT/EiNlg6m\nLg7RWS01toalDPJJZ1iedLihq2b9cwDqAZyplFqrlLpIKTWEeNDeDeAKi+vpVUrNUEodq5S6UCl1\njlJqKYCTAVTZWE/eWNi5vybZ7ogCgM8165o2ZlburljXi3ERn1J4WHlJIbYOTsSc1trsC7vgLoAb\nmRNuL5hObzxTvzZ8nPl1bNu54c1UBqyMP75loCVjnmXag67y2udgSE/fvMxPEq7dMhO/umgowzJe\nbV0Dwzbr9rboNEczdR5OtjKlf4CO5otmawhbMOb5BHLaNpCpjGT+5tgK/ycda68rz7pMmCfIcsN1\nsC4ibQBWAtgB4IaUjz8C4E0AR4tI1r2slDKbaeD2xGt4ByzVxG4Z7KivwJWb+nD47GbctG2O7e1V\n+NgOUWdb8+u2zMza0z0qgjy1tLtot2mWm2ZDgXnFTvtdL9gJ6jKVgVuON+5YNWzb/Nas07tnWr+r\npygmq80aIFrYptEi2W7szftaJK8jdZ32tuM6WPewYKce89nywU5SwvxAIGxBuRmvb27trN1q86jU\nY2iUh2PuO/WlY2anzbqbim3WzQ1XTdyplBoxFZZS6nUAvwJQBmAw9Ys2HJp4fcjFOnLWxv5mfGpj\nn6MOM6VFBbjmiPTOR2F3aF8jHvjgihHvhbGMhvna0ja23NU+22vy467fMsvRMeX0yYufF3BBeuDj\n12Fnpa13pl1R50FNWFBlzrwZjPl3bNc8uzyujpnX6m4FCZZHt/C6yZy3q08TlcDcSKbRd8IkOU9T\nbzCsTCxmK4+sHEBZlmmrq8CXjpmdcZkcrVjXEqx3J17/avL544lXywOvisiJInKpiFwpIv8D4GsA\n/gbgIufJJDOHzWjyZTvamsHsmx0xN0plUD+jvKTQ3QXY5ExdXVbk6Jiy+uTlo4dNHfk9jbdEZy3P\n/PDO6JizcxxmWrYw5v50nGkXbh2cuK8zZLYaejvrDZMRbdbTPvM3Lecc0IVN/c1YP7MJc1otj7Ng\nSa/BiDq2f59JnnaO82b8cbdDN8bXEf4DcWpjFTbM8nY8jHh5tLYvMu3z5JGISgsL0JIYdrW9rjzt\nhuOqzX02UzlSVan7oUGHfeTQKaafeTm+fZB0BOvDZ43XTD4ffj/zlHYjnYh4E5rzEG9i8yCAFUqp\nxzN+K0FEHjT6A6CnC7vPCnP04HPK6QREXjJPk/1EReUexM4v2za/Vdt2U2sszWr4nTjngC7bY2fb\nya5s45kPtsX7KBw41VnnxoYMNXqlRQW4+/yl+NVFQ9gy0GK6nBGnTY2sHMtO1mypki7gglRRUohP\nb+rDZzfPsByozMwy8ysAnL2iE9Oaq0dcFybUZj5m7eyLqtIifO34AWwdtHeMANnzxW2nUd3neC+O\nkR+esdByu/4gfPCQ+CR8JYUxXLhqf0gUiwlu2jYHFxzYja8cOydt36yb6e4GZNv8VtSUxcvBh1ab\nB9tWHLdgEn523hLDc7VRM5go3ORl40eD5eE9Z3lvKaUGASAxiswsxDuWPigim5VSP9GfRPKDmwlE\nwsCrm4GgRuRRUNqHbjTT01CJtTMacccfX3C8PTN2A8n1M5vw3T8872hbsu8/Se9pzL5bjp+Lh59/\nFX3Nduo29qsoKcTnj5qFHzz0Ao4zGP6zuDDmqByG7VJnJT1us0Vnvlrdf8t76vGHZ17NuMzw7KB3\nbF+AW369Awf1jkdlaRFef/s9d4lMsqSrDku66nDb/c9oWyeQnid2KzPC2sG0pDCGd3bvxaWHejOU\nb6oxFcV4/GVn393Y34wFHWNRV1mS1km0o74CHfUdpt/tqK/AEy+/AQBY2m1vmN7ykkLcc+EyPPfK\nLkxprMLHfvh/6QvZyMe2ugqUFqXfFOVq3aaOYH245tysgVNVynKWKaV2AvipiPwOwF8A3CIiE5VS\nu7J8z3Du5kTtunfTlnkkbBfKYXbPSVMaq3D04ETcev/f9r0XlgmO3Irqz3BzXrNTW+FpZzuTqnWz\ni2aVhpkaR2xH02gwgngw3T/R3QhAB00bj4OmjXe1jjRmHUwDmxjN/nfsJrWsWF9dlhedoHubqvGp\njc6bJng1AZ/p95W9zn9Gu6yixLjT49jKYAcbuOfCZXjuX29hVove5k7JrljXiw/d8Qi6G6qwenoj\n7n9qp6XvGZXRyePtzZg77Maj+/GhOx5BS20ZNs+ZgBdf2x+K1VVm7xNTVVqEKY3Zz7+H9jXiB39y\nVrHDDqbmHku8mrVJH24EatamPSul1KsAfg2gDsDULItTiH1sba/rdfgRGHdpGD/aKieTUehQW+6u\nw2G2fEgerWe25iEo+5LG913Q4d/oM4bXAZ+uDbqP+96m/Rfs+SaTDO3btkmoFubLopNytWJyvLaw\nvrIEGx1M+GQm6Jv5sMQvbtMxq2X0vqZCZw7trwEeXz0K71vWgcbqUnz2cOs3ME6SYxT8jqsqRf/E\nWk9vXo+aOxG/vWQF/vuMhYENT9heV4GvnzSI/9gwHQUxQfPoMlxzxAxs7G/G10+c634DiZ91aYY2\n6dkYNoOJaGVaMh1VB3cnXleKSCx5RBgRqQSwAMAuAPe73M5wj7XdLtcTakYHVUNVKZ5/NePDhMgK\nul2pGbud/Uwfz7pPilarpjbgJ3/+OwBgWXedq4tneZaax6+fNIjr734C89vHoL2uwvG+MCoTNxw1\nC7f++m8YbBtjqUZHp9Sgz84+7JtQbVqW/S4K12+ZhZNvfQBlxYW4JNGO1YzZxW5ZTz0+mnicPbUx\nvbbOq59kZV85uek9bWkHzjmgCxPHlOOlf5uNJGyfm5p1q8dF2M41RtzWeooIvn3qfLzw6i5MqB05\nr8j5B3bj/AO7Tb6pT5BPgzONbb56+nj88KEXs65Dd2XQYTOatA1S0TY2PsL3mIoSLOoci3sf/2fG\n5Y3ih1wdZ911sK6UelJE7kS8I+h2ANclfXwZgHIANyql3hx+U0R6Et/9S9J7EwEUKKWeSt2GiJwC\nYA6AZwE87DbNYVad8oheJP7o6a5HX8bN9z2N7cvM25NFUViawXhVvN3+vE39zfjt068A2F/rBzgP\n7C5f14uCgvjpet3MJvPxsy2s/+Pre7His/fs+3fqurobKnHdFm8mHW4eXYaLD44HmI88b7uFnWNG\nFzo7WXHZml488vy/seu9PbhszVSc/p+/15c4m1rHluN/zl5s6YbZ7DCeNLYc126ZiQd2vIKTFrXp\nTWCm9FiaGTjzv82+MzUx0spLFtOi8ybLq8oLo7UGceq1UwdilryCmKQF6rmmsrQQpUUFOGPI+vX+\nui0zDYP1qISuR81t0fIENqT1f67papR3OoD7AFwrIssBPApgLoBliDd/uSRl+UcTr8m7dSaA74rI\nfYnvvARgDOLjs08D8AaAo5VSezSlOZRaxpRh/cwm/OChF7B+ZjPeN9SBCbVl6G2qxpnLO0JbE+0n\n3dcYqzO/ZpxaWWOikrN4w6xm7Nj5Jl769zu4MKnWyOmFdmxFCW44cn+3jTffMX5Q1dOQffi2jnpv\nhnizy3zCHn/YKZN1lSX4+flLsUcpPPvKW47Xo4vVbWaqGV7T14g1fY26kuQZu8M8W80NK2XRj8A4\nJPUephTSa9YH28ybXy1Kat42XOMaBn6U0zOHOnHiokm2h4VtHVOGHTvfyr5wyDTVjMIV6+wNJ2sm\nV9usawnWE7XrswF8FMAqAAcDeBHAtQAuU0q9YmE1vwdwFYBFAA4BUAvgbQBPAfgMgGuUUs/qSG/Y\nfXbzDHx8/TSUpswgFsTF3Oshj5z8Jt0XpS8c3Y+Hn8teOzu7tRanLmnHr5/auW/4Kx0y7YJYTHDB\ngf6NOLp1sAXLe8ahpiy9w9aWgQn4xm/jRXDz7Am21+1VMGEWSJqPHOEiIWLUFtqeWEwQM/hWUUF4\nLzK68u4LWw37/o9w2Zrs3ZJ0ngoP6m3Ajx/5O5pqRmG6w1F4svFllt0QBOt28+XyDH2YWseW4+rN\nM/Cbp3fi5MXtLlMWPbqu91GIXXWm0agZTNhvZK3Q1t09EUgfZ3HZtL2plHoG8XHVCUgL1IPQXleO\nhqr0cZuHC5aOAhZ0M5j+iaPR01CVFqyPqTAeXeCig7wPnK2NIe3Nti9fa167cdGqyfj3rt1QULj4\n4PBMWRDV83DqxThbP4kgxwrWVU6tjB9/rIUx+Z01gzEuNJ/aOB0HTm3A3LZaR+1drZTFs1d04b4n\nfw0AOHWJN4Gn7dFdPDicxmSZZTe11rPe4PqSbO3MJqyd6c+kfXkjAsE74Pz4NCrCfg4Y4RU/xlmn\niJnaWIVj5k3Eos66fRe4NX2N+H9/egFNNaM8HZ7Kb0bjtALAx9dNw3U/exy3P/AcAODw2U5Hhsh+\nxglqnHXAXtBfXVaEG44K38inZoGk2U9zE6MYDgajKfvCPPmZ487BKf/28+lg6rbMtlxZWmQYENZm\nCTztGJhUi+uPnIkXX30bR861N9mQ1fODzqEm7fji0f04+dYHUVwYw2WHZRjtS6nQdf6LQq2zGaPT\nXlQrLnSqqxx5Azi+utT1JExhwGCd0oytKMHmOSMvKJ/cMB0H9Tagv3W01hOus4u38Smp0cFkL2YX\nwgm1ZfjAwfGaZAD4wMH6mr2QPdlqDO0GKW5rFFOPGKc3W3tSxocP8zTZUXyMnJZPNndvTVnxvkoK\nHVZPz96u/8iBFlzzv4/j3T17bc/6W1wYwxe2zsKpt3nUadnkGFg5tQE/P38pqkYVZb3BCdsRnjzj\n7+gyvfMvhEWQlUFuWHlSZPTL+ieOxorJ43DPX/+BC1d1Y9v81lDPKGsVg3VKYxQzjCou0D/RCtw/\nXv/KsbNx1f/+FaunNzoK1itLzYtATVkxvnB09ja2gPmJRefj+qjLtiucHgndDZVY2DEWv3zin7ZG\nT9DFaXalButh5rScWtk189rH4L8To1j0ZerEHYCTFrVpC9atGF1ejO+ePh9/fuE1HGIhuE+1qnc8\nSotiePu9vVmX1dmsqtViB9CwndtKCgvwndPm4SeP/B2HO+iHEzbh2rveO21pO869/U9p73/52NnY\n9e4ejCoOvjmxLgzWKY2u3tTjqzO3R9Rh+eRxWD45ezvYZFdt7sM53/oTCmOy7/FY2C4ifolCz3kr\nNUO3njCAl19/B+OS28CafK3Lwkg3pmnRuLt2780eUIWFrmYwRi49dCqefPkNvLt7L645Qt9Qn37d\nBE9v2t8pdaLFkaUy6W2qRm9TuG5aAGdPLpPFR4PRkxad+ifWup41OCyMylsETvGOHTajCR+64xG8\n+W76IIG5FKgDemYwpRxjtXCbBVGFMcHUxips0jgDYDK3j+TXzWzGD89YiF++f8j1BShbmuYlzQw5\nY4LxSBPpzSr8U1pUgAUd8TSusHnTY9fCjv37osnGfrf0OFRkZKCeweLOsdjU34yW2jJcuMreJCoC\nSW8L7VPNepBNUbx8CFBXWYIfn7UId523xHINbZhUlxXhthPm4oSFk3DzcQPa168j2HK6jq+fNBeF\nMcGoogJ8Yr37ofWiUDmQ63IpCwpigpMW+ze/Q5BYs05pnNZAXbtlJuZOqkVZcQEqSgpDXVvtV81V\nTVkxvnnyIO574p84YsBexzK/fHXbAB5+/lX0uRy2LlswubF/Ah7Y8S88/c838XENF/5s1s1swo2/\nSJtjDQDw6U3xKckfeu5VfAqPudqO0zahu/OgGYxVYT5XWLGwcywWdo7NvqADOna9YWdEC+ud3z4W\n9100hJKigrQJ+5ywOTE05bEo9pPxEoN1SlPg8MIZhYlRvJLpvDLYNibj5B9BByrFhTFfHgMXxGRf\nkGyH05N2T0MVrtsyE488/xpuvMc4aNdxQXCafbv38GrkVNBDvkaNm1NMtuEV7aUj2jdlQHjahW9f\n1oFv/u7ZxP9Hbxz6pppReP7VXQCA3sbwNfsKG97nUpqwn09z/TId9v0fJYf2NeLilJF8kuM8u8eS\niL5mS7Xl0Rl9wmls7FVM7VfQF4ayqCMNBQZV2ueu7HK/YhuUYjOYmqQRZ+ZMcldBMqG2DN8+dR4+\ntWE63res023SfPflY2djbEUJmkePwkfXZp8ILd8xWKc0Vk+oXp13J4+vyvh5GGvV3KQpvy9f2Xl5\nfTfKt0zbMx5n3VkCO+orsWFWM0YVFWScyTEMgpyQKUhTxlehpTbeafSg3oaAU+Nc6hj+t54wgJ6G\nzOdZ3RRUKDuY2nWei5ucb5w0iAOnjsOHV0/BVA21ybNba3H4nAkZO1O6nXHZK5PHV+G+i4bwiwuW\nob7S+8Eooo7NYChdWEpzwsb+Znz7weeCTgZ5zSQe9PLezGjozkzbMwrM3RSXzxzeh09umGZpHOAg\nw+Ww3R/7dcMeiwm+fdo8/OapVzDUU+/LNr1QlHJ8LeqsCyQduVCz7uY4mDy+CjcePTvrcm110eto\n7URxofl5z2oRnxbCkZO8wJp1ShP0CTX1Qnylg3bOdrn9xW4ey+fLOOth1FFfieWJi+/ZK6w9Sk7P\nL3dpiMKEHRHqC6tdfWUpDu1rRHlJdOu2CgvCcU7JhVObiGCSB6MWTagdhYaq+LEW5RtDvw311GPL\nQAt6GirxX6fOCzo5nonu2Ye0OmDKOPz0/14CAGwO+eQQYYwb/G6aYzYCSaRP8j5dyFNz6svHzsa/\n3npv3+yL2QKK1KzOh5ursDWDsTpMZz4yOjcUhWAYloaqUYFXBIXZkq46fOywXu3nk6jOYGqViGgZ\nVjTsgi/B5KuZSWN9jyra387tinW92Da/FZceOsWzIcisyofgJ5mIYEtiWMcj51ob3tEoeDqotyHa\nJy2f4sHU9rsiknWa9H3LIpw3i14LQzOYr26bg9kTR+Nja3sx2iC/6ipLRvw714MUO4KqWb/l+AEU\nxATlxQX42NqpDNYzUMqfa1++XV9zBWvW88zR8ybi3sf/gR0738LVm2fse7++shSXrglHj2yjWurt\ny9pxw91PAgDOWh6+nu9uY5lPrJ+G81d2YUxFSfaFDbSOKcPnt/a7TEU4HTm3Bf/5m2cAxEd3ceLw\n2c24/YHnMKd1NCbUup9lkvy3rKceyxJPjl7b9d6Iz05YOAlbBsL9RDBIqTeoflncVYf7LhpCWXEB\nKkuLOFypx4xuqhmb5wYG6wFZPX08fvjQi75vt6gghq96MMue17Yv64BAUFIYw1FzJwadHE/YCdTD\nWGvoVVOJqY3VuOHIWXjspdexbX6ro3V8csN0HDu/FV3jKt0lRsI5GpHX9jpttO7Z0I0j//3+VT1p\nndXC1nR/2CIbAAAVNElEQVQnSEH2i0husjShtgxnLe/ED/70As4/0N7swbmOR+tILL8jMVgPwFFz\nW3DZmqkjgvWyDEMvhVVnfQX+tvMtAEClx52vyooLQ31yz8P4zVeHTB+PQzDe8fdFRMtQaYC/F9Wt\ngy247f74U4WtFptIeYGHd3B0NFsoCkkHUwA454AunHOAv2O85wvWoucuBusBmN06Oq2moyKCIw1c\nvnYa/vDMvXj7vT342gnRq60nsmLb/FbcfN8OAMCJC9t83fb7V/Wgu6EKU8ZXaZ1J0i6nN6MlRf7U\n6BoFKWF8+uSEjic55cWFKCoQvLdHYYzF/hlkzounazpWadgMxv1qA8EKsJGiFyHmgFw5CBuqS3Hf\nxUN4d/deVJbqm42xJMPYq2GVI1lKBs5d2YXde/eipLAApyxpw+tv7/Zt25WlRTh6MPhmX04fSZcW\nFeADB/fga/f9DadHcEr0KDp42nh85/fxeSlWTB4HID5e/B3bF+BHD7+Iw2Y0BZk8MuXPVSSqwXu+\nY7BOrpQUFqCkUG8Tnk9unI5VV98LALhqs/djrBNlUlVahMvX7h9lx89gPSzcVDCcvLgdJy/WG6hb\nCThypc2r3WYwHzxkMl5+/W28t2cvPr5u/8y4UxurtTUFI/28qsTj6C+5gcE6hU5PQxV+eMZC/Out\nd7GgPdhhJC3z+XFJ0+hRIzfv69b3++hhU/Hh7/858f+9WZamqMqNsDc/jC4vxq0nzA06GWRBcWEM\n7+7eCwCYkTSsMlEqBusUSr15MoWwU2ev6MRXfvl00MnAloEW1JYXY1xVqfuRVkIi66RIeRi6RnEE\nnFxps07hU19Vih2JwRXcuP2Uebjgv/6EznEV2BTyyQj9Fr0zjrcYrAcggtc9ysbnR406+wi4UVQQ\nw+rpzsY+TxWWIDhr+QxHMn0Vtk6JqY/2jUpfUBMBUe771IbpWHnVPdi9dy9uOd75U4wZE2rw03OX\naExZutRSwFYx0RS9nnxEHnB9AuMdGOWwLXNb0FJbBhHgkxuiMUvu2IoSLOyIN6NbPyu6nSoZW4VP\n69hy/PriIdxz4bLAZ/zOWbykjsCadSIKhbA0W8jeDCb/lBQW4K7zlmDnG++ioTq4ISTt+trxA3j8\n5dfRHeEmWtWjwvEUjUZyOtu031iTnhtYsx4SLFDRlo8BnG5haQaTTb4+RCkqiIUmULd6uiyICXoa\nqiI9IsaizrH7Oh9efFBPwKmhMJvZsr+TakttWYApId1Ysx6APL3W57R57WPw0HOvAQDaxpb7vv18\nDSC9kG1fVo3iaTNsohyMZyMi+O5p8/HPN94JdGIsCr8Pr56Ch59/Dbve3YMvHTM76OSQRrzqEGlw\n1vJOPPzca9j5xru4/siZQSeHPFRWXIhPbpiG2x94Dict8ndGU8pPsZgwUKesxlSU4K5zl2DPXrVv\nlvT0ztjRuLGNypNWvzBY98n46lK8+NrbAICB1tqAU0Op3FbMlRUX4usnDepJDAXKyrGweU4LNs9p\n8T4xZCiHK9KJXBGRjCMhMQiOJrZZ98mtJwxgU38zrt48Ay1j2JaMiIiIyEhH/f5O4bw5Z826bzrq\nK/HpTX1BJyN0RhUXBJ0EIoo4XsuJcstFq3pw7+P/wGtvvYevbJsTdHICx2A9JKLSjkyHs1d04ur/\nfRz1lSVYOzO64x8TERGRftVlRfjFBcvw7u69rNQDg/XQyKd2ZGct78RQTz3a6ipQUshCqEM+HT9E\n+VS5QZSvCmLCQD2BwTr5TkQwvbkm+4JERESkDYf5jSZ2MA0J1hSRG7lw/PAiQk6xAxoR5TIG60Q5\ngM1gKJ8wOCeifMJgnQi5UTMddQzAiIiI0jFYJ6JQCEszGN64ERFRmDBYJwJQWhTtHudhCXRzAZsU\nRU/qlOpEZIxnt2hisE4EYMXkejTVjAIAnLqkPeDUEBEREcVpG7pRRJoBfBTAKgBjALwI4A4Alyml\n/mXh++UA1gI4BMAsABMA7AXwGIBvALhOKfWurvQSJSssiOHOcxbjiZffwPTm6qCTYxsrFvVhMxgi\nylU8u0WTlmBdRNoB3AegHsD3AfwFwACAswCsEpEFSqmdWVazCMBtAF4BcDfigX4tgEMBXAlgvYgs\nV0q9rSPNRKnKSwrRNyGa47+zGQwREWXDS0U06WoG8znEA/UzlVJrlVIXKaWGAFwFoBvAFRbW8XcA\nWwGMV0ptTKzjZABdAH4PYD6A7ZrSGzqsGSUKh7Uzm/b9/4rJ9QGmhIiISEOwLiJtAFYC2AHghpSP\nPwLgTQBHJ5q5mFJK/VEp9Z+pTV2UUq8D+Ezin0vdpjesWDNKFA6nL23H+llNWDllHD6+blrQySEi\nojynoxnMUOL1TqXU3uQPlFKvi8ivEA/mBwHc5XAb7yVedzv8PhGRJaVFBfjs4TOCTgYRkRYzW2rw\nh2deRfe4SpQXR3vks3ylI1jvTrz+1eTzxxEP1rvgPFg/PvH6EysLi8iDJh/1ONy+59gMhoiIiHS7\ncWs/7vy/lzDUU89hTiNKR5v14aEzXjP5fPh9Rz33ROR9iI8w80cANzlZR1itmtqw7/+T28kS5SM2\nBSMi0q++qhRbByeiMTE8MUWPtqEbMxi+jbN9KRaR9QCuRrzz6Qal1HtZvhLfkFL9Jut7EPFhIUPh\nY2t7oaBQXlyIM4Y6gk4ORRgDXSIiotykI1gfrjk3G5y6KmU5S0RkLYBvAngZwDKl1FPOkhdedZUl\nuPHo2UEng4goUvgkn4jyiY5mMI8lXrtMPu9MvJq1aU8jIpsA/BeAlwAsUUo9luUrREREREQ5R0ew\nfnfidaWIjFifiFQCWABgF4D7raxMRI5EfMbSFxAP1B/XkEYiIiIioshxHawrpZ4EcCeAVqRPWnQZ\ngHIAtyil3hx+U0R6RCRtZBYRORbArQCeAbA4F5u+EBERERFZpauD6ekA7gNwrYgsB/AogLkAliHe\n/OWSlOUfTbzua3koIssQH+0lhnht/XEGQwy9qpS6WlOaiYiIiIhCTUuwrpR6UkRmA/go4sMsHgzg\nRQDXArhMKfWKhdVMxP6a/uNNlvkb4qPDEBERERHlPG1DNyqlngVwnMVl06rMlVI3A7hZV3qIKFo2\n9DfjJ3/+OwBgSVddwKmhMCuK6ehuRUQUDX6Ms05ElNWKyfV4/6oePPPKmzh7hdngUkRALCa49YQB\n3P7Ac9gyMCHo5BAReYrBOhGFgojgtKXtQSeDImJRZx0WdfIJDBHlPj5LJCIiIiIKKQbrREREREQh\nxWCdKAcopYJOAhEREXmAwToRERERUUgxWCfKAQYTiBEREVEOYLBOlAPYDIaIiCg3MVgnIiIiIgop\nButERERERCHFYJ0oB7ARDBERUW5isE5EREREFFIM1olyAMeCISIiyk0M1olyAJvBEBER5SYG60RE\nREREIcVgnYiIiIgopBisE+WA0WXFQSeBiIiIPMBgnSiibjl+AAAQE+CqzTMCTg0RERF5oTDoBBCR\nM4u76nDXeUtQWlSApppRQSeHiIiIPMBgnSjC2usqgk4CEREReYjNYIiIiIiIQorBOhERERFRSDFY\nJyIiIiIKKQbrREREREQhxWCdiIiIiCikGKwTEREREYUUg3UiIiIiopBisE5EREREFFIM1omIiIiI\nQorBOhERERFRSDFYJyIiIiIKKQbrREREREQhxWCdiIiIiCikGKwTEREREYUUg3UiIiIiopBisE5E\nREREFFKilAo6Db4RkZ2jRo2qnTx5ctBJISIiIqIc9uijj2LXrl2vKKXGuFlPvgXrTwOoArAjgM33\nJF7/EsC2KY55EA7Mh+AxD4LHPAgH5kPwcjkPWgH8Wyk1yc1K8ipYD5KIPAgASqn+oNOSr5gH4cB8\nCB7zIHjMg3BgPgSPeZAd26wTEREREYUUg3UiIiIiopBisE5EREREFFIM1omIiIiIQorBOhERERFR\nSHE0GCIiIiKikGLNOhERERFRSDFYJyIiIiIKKQbrREREREQhxWCdiIiIiCikGKwTEREREYUUg3Ui\nIiIiopBisE5EREREFFIM1j0mIs0icpOIvCAi74jIDhG5WkRGB522qEnsO2Xy93eT78wXkR+JyCsi\n8paIPCQiZ4tIQYbtrBaRn4vIayLyhoj8RkSO9e6XhY+IbBSR60TkXhH5d2If35blO77saxE5VkR+\nm1j+tcT3Vzv9rWFmJx9EpDVD+VAi8s0M27G1T0WkIJG3D4nIrkSe/0hE5uv43WEhImNE5EQR+Z6I\nPJH4ra+JyC9F5AQRMbyGsizoZTcfWBa8ISKfFJG7ROTZpN/6BxH5iIiMMfkOy4IGnBTJQyLSDuA+\nAPUAvg/gLwAGACwD8BiABUqpncGlMFpEZAeAGgBXG3z8hlLqypTlDwPwHQBvA/gWgFcAHAqgG8C3\nlVKbDLbxPgDXAdiZ+M67ADYCaAbwGaXU+bp+T5iJyB8B9AF4A8BzAHoA/KdSaqvJ8r7saxG5EsB5\niTR9G0AxgCMA1AI4Qyl1vfNfHT528kFEWgE8DeBPAO4wWN0jSqlvG3zP1j4VEQFwO+J59RiAHySW\n3QygFMAGpdT37f/a8BGRUwF8HsCLAO4G8AyAcQDWA6hG/JjfpJIupCwL+tnNB5YFb4jIuwB+D+D/\nALwMoBzAIIDZAF4AMKiUejZpeZYFXZRS/PPoD8D/AFCIHyzJ73828f4Xgk5jlP4A7ACww+KyVYif\nTN4BMDvp/VLEb6AUgCNSvtOK+EllJ4DWpPdHA3gi8Z15Qe8Hn/b1MgCdAATA0sRvvy3IfQ1gfuL9\nJwCMTlnXzsT6Wt387rD92cyH1sTnN9tYv+19CmBL4ju/AlCa9P6cxDHwMoDKoPedpv0/hHhwEUt5\nvwHxgFEhHpANv8+yEI58YFnwJh9KTd6/IrEfPpf0HsuCxj82g/GIiLQBWIl4gHlDyscfAfAmgKNF\npNznpOWLjQDqAHxTKfXA8JtKqbcBfDDxz9NSvnM8gBIA1yuldiR9518APp7456leJThMlFJ3K6Ue\nV4mzXhZ+7evhf1+RWG74OzsQL2MlAI6zkN7IsJkPTjjZp8N5+cFEHg9/53eI14TVIX5MRJ5S6mdK\nqR8opfamvP93AF9I/HNp0kcsCx5wkA9OsCxkkfwbU9yeeO1Meo9lQSMG694ZSrzeaXCCeR3xO/Ey\nxB8hkXUlIrJVRD4gImeJyDKTtm/D+/8nBp/dA+AtAPNFpMTid36csgzt59e+Zv5Y0ygipyTKyCki\nMj3Dsrb2aSIP5yOep/da+U4Oey/xujvpPZYF/xnlwzCWBX8cmnh9KOk9lgWNCoNOQA7rTrz+1eTz\nxxGvee8CcJcvKcoNDQBuTXnvaRE5Tin1i6T3TPe/Umq3iDwNYCqANgCPWvjOiyLyJoBmESlTSr3l\n5kfkGM/3deIJVBPifRNeNEjD44nXLhe/I1cckPjbR0R+DuBYpdQzSe852acdAAoAPKWUMgqO8iIf\nRKQQwDGJfyYHCSwLPsqQD8NYFjwgIucDqEC8v8BsAAsRD9T/I2kxlgWNWLPunerE62smnw+/X+ND\nWnLFVwEsRzxgLwcwDcCNiLdN+7GI9CUt62T/W/1Otcnn+cqPfc3ylN1bAD4GoB/xNp6jASxBvEPe\nUgB3pTS78zLfcj0f/gNAL4AfKaX+J+l9lgV/meUDy4K3zke8Oe/ZiAfqPwGwUin1j6RlWBY0YrAe\nHEm8cjgei5RSlyXaLr6klHpLKfWIUupUxDvsjgJwqY3VOdn/zDNn/NzXeZs3SqmXlVIfVkr9Xin1\nauLvHsSf4P0G8ZrAE52s2sayOV9GRORMxEee+AuAo+1+PfHKsuBSpnxgWfCWUqpBKSWIV5ytR7x2\n/A8iMsvGalgWbGCw7p1stbBVKcuRc8MdjBYnvedk/1v9zr9tpS73+bGvsy2frYYlbyUe0X858U87\nZcRon+b1eU1EtgO4BvGh65YppV5JWYRlwQcW8sEQy4JeiYqz7yF+EzQGwC1JH7MsaMRg3TuPJV7N\n2koN95o2a9NO1r2ceE1+rGm6/xPtHCch3iHpKYvfGZ9Y/3Nsr57G832tlHoTwPMAKhKfp2J5ymz4\n8fS+MuJwnz4BYA+AtkTeWvlOThCRswFcD+ARxANEo4nYWBY8ZjEfMmFZ0Ewp9TfEb5ymisjYxNss\nCxoxWPfO3YnXlZI+u1olgAUAdgG43++E5aB5idfkQv+zxOsqg+UXIz4Sz31KqXcsfueglGVoP7/2\nNfPHueFRp55Ked/WPk3k4X2I5+kiK9/JBSLyfgBXAfgj4gHiyyaLsix4yEY+ZMKy4I3GxOuexCvL\ngk6pA6/zT98fOCmSzn05FUCtwfsTEe/xrQB8IOn9KsRrUOxMyDAJnBTJaN8vRfZJkTzf18iTyS9c\n5MNcAMUG7w8l9o0CMN/tPoW1iWCqgt5fGvf7hxK/9wGjc1DKsiwL4cgHlgX9+78HQIPB+zHsnxTp\nV0nvsyxo/JPEjyIPiEg74gdlPYDvIz480VzEZyX8K+Ini53BpTA6RORSABch/sTiaQCvA2gHcAji\nhf9HANYppd5N+s5axKcefhvANxGf6ngNElMdAzhcpRQAETkDwLWwMdVxLkrsu7WJfzYAOBDxmqjh\n8YT/mbwv/NrXIvIZAOdi5LTSmxFvL5kb00onsZMPiSHppgL4OeL7BwCmY/8Ywx9SSl1usA1b+zRl\nivW/ID7F+hjk4BTrInIsgJsRry28DsZtX3copW5O+g7LgmZ284FlQb9E86NPIz5G+pOIH6vjEB9l\npw3A3wEsV0r9X9J3WBZ0CfpuIdf/AExAfMjBFxE/6P6GeMeYjDUD/Evbj0sAfAPxE+KriE+E8Q8A\nP0V8nF0x+d4CxAP5fyHe7OhhAOcAKMiwrUMB/ALxG4I3AfwO8XF5A98PPu7vSxGvrTD72xHUvgZw\nbGK5NxPf+wWA1UHvs6DzAcAJAH6I+KzJbyBeo/UM4he8RTr3KeJzdJyTyONdiTz/EVJqK6P+Z2H/\nKwA/N/gey0KA+cCy4Eke9CI+I+gfAfwT8fbmryX21aUwiWlYFvT8sWadiIiIiCik2MGUiIiIiCik\nGKwTEREREYUUg3UiIiIiopBisE5EREREFFIM1omIiIiIQorBOhERERFRSDFYJyIiIiIKKQbrRERE\nREQhxWCdiIiIiCikGKwTEREREYUUg3UiIiIiopBisE5EREREFFIM1omIiIiIQorBOhERERFRSDFY\nJyIiIiIKKQbrREREREQhxWCdiIiIiCik/j9yTL/cJf6jGQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1aa101b358>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 250,
       "width": 373
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(losses['test'], label='Test loss')\n",
    "plt.legend()\n",
    "_ = plt.ylim()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用测试数据进行测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "ffm_test_out_path = './te_ffm.out.logit'\n",
    "fm_test_out_path = './te.fm.logits'\n",
    "test_path = './data/test.txt'\n",
    "\n",
    "ffm_test = pd.read_csv(ffm_test_out_path, header=None)    \n",
    "ffm_test = ffm_test[0].values\n",
    "\n",
    "fm_test = pd.read_csv(fm_test_out_path, header=None)    \n",
    "fm_test = fm_test[0].values\n",
    "\n",
    "test_data = pd.read_csv(test_path, header=None)    \n",
    "test_data = test_data.values\n",
    "\n",
    "pred_test_X = np.concatenate((ffm_test.reshape(-1, 1), fm_test.reshape(-1, 1), test_data), 1)\n",
    "\n",
    "# pred_test_X = cc_test[:,0:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_params_with_name((pred_test_X), \"pred_test_X\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pred_test_X = load_params_with_name(\"pred_test_X\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1000000"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(pred_test_X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_tensors(loaded_graph):\n",
    "\n",
    "    dense_input = loaded_graph.get_tensor_by_name(\"dense_input:0\")\n",
    "    sparse_input = loaded_graph.get_tensor_by_name(\"sparse_input:0\")\n",
    "    FFM_input = loaded_graph.get_tensor_by_name(\"FFM_input:0\")\n",
    "    FM_input = loaded_graph.get_tensor_by_name(\"FM_input:0\")\n",
    "    pred = loaded_graph.get_tensor_by_name(\"inference/prediction:0\")#\n",
    "    \n",
    "    targets = loaded_graph.get_tensor_by_name(\"targets:0\")\n",
    "    lr = loaded_graph.get_tensor_by_name(\"LearningRate:0\")\n",
    "    return dense_input, sparse_input, FFM_input, FM_input, targets, lr, pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict_click(x, axis = 0):\n",
    "    loaded_graph = tf.Graph()  #loaded_graph\n",
    "    with tf.Session(graph=loaded_graph) as sess:  #train_graph\n",
    "        # Load saved model\n",
    "        loader = tf.train.import_meta_graph(load_dir + '.meta')\n",
    "        loader.restore(sess, load_dir)\n",
    "    \n",
    "        # Get Tensors from loaded model\n",
    "        dense_input, sparse_input, FFM_input, FM_input, __, _, pred = get_tensors(loaded_graph)  #loaded_graph\n",
    "    \n",
    "        feed = {\n",
    "              dense_input: np.reshape(x.take([2,3,4,5,6,7,8,9,10,11,12,13,14],axis), [1 if axis == 0 else len(x.take([2,3,4,5,6,7,8,9,10,11,12,13,14],axis)), 13]),\n",
    "              sparse_input: np.reshape(x.take([15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],axis), [1 if axis == 0 else len(x.take([15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],axis)), 26]),\n",
    "              FFM_input: np.reshape(x.take(0,axis), [1 if axis == 0 else len(x.take(0,axis)), 1]),\n",
    "              FM_input: np.reshape(x.take(1,axis), [1 if axis == 0 else len(x.take(0,axis)), 1])}\n",
    "    \n",
    "        # Get Prediction\n",
    "        clicked = sess.run([pred], feed)  \n",
    "#         print(np.array(clicked))\n",
    "        return (np.int32(np.array(clicked) > 0.5))[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0]], dtype=int32)"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict_click(pred_test_X[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [1],\n",
       "       [1],\n",
       "       [0],\n",
       "       [0],\n",
       "       [0],\n",
       "       [1],\n",
       "       [1],\n",
       "       [0]], dtype=int32)"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict_click(pred_test_X[:20], 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_test_batches(Xs, batch_size):\n",
    "    for start in range(0, len(Xs), batch_size):\n",
    "        end = min(start + batch_size, len(Xs))\n",
    "        yield Xs[start:end]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict_test(batch_size, axis = 1):\n",
    "    loaded_graph = tf.Graph()  #loaded_graph\n",
    "    with tf.Session(graph=loaded_graph) as sess:  #train_graph\n",
    "        # Load saved model\n",
    "        loader = tf.train.import_meta_graph(load_dir + '.meta')\n",
    "        loader.restore(sess, load_dir)\n",
    "    \n",
    "        # Get Tensors from loaded model\n",
    "        dense_input, sparse_input, FFM_input, FM_input, __, _, pred = get_tensors(loaded_graph)  #loaded_graph\n",
    "        test_batches = get_test_batches(pred_test_X, batch_size)\n",
    "        total_num = len(pred_test_X)\n",
    "        \n",
    "        pred_lst = []\n",
    "        for batch_i in range(total_num // batch_size):\n",
    "            x = next(test_batches)\n",
    "                \n",
    "            feed = {\n",
    "                  dense_input: np.reshape(x.take([2,3,4,5,6,7,8,9,10,11,12,13,14],axis), [1 if axis == 0 else len(x.take([2,3,4,5,6,7,8,9,10,11,12,13,14],axis)), 13]),\n",
    "                  sparse_input: np.reshape(x.take([15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],axis), [1 if axis == 0 else len(x.take([15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],axis)), 26]),\n",
    "                  FFM_input: np.reshape(x.take(0,axis), [1 if axis == 0 else len(x.take(0,axis)), 1]),\n",
    "                  FM_input: np.reshape(x.take(1,axis), [1 if axis == 0 else len(x.take(0,axis)), 1])}\n",
    "    \n",
    "            # Get Prediction\n",
    "            clicked = sess.run([pred], feed)  \n",
    "            pred_lst.append(clicked)\n",
    "            if ((total_num // batch_size) + batch_i) % show_every_n_batches == 0:\n",
    "                        print('Batch {:>4}/{}   mean click = {}'.format(\n",
    "                            batch_i,\n",
    "                            (total_num // batch_size),\n",
    "                            np.mean(np.array(clicked))))\n",
    "#         print(np.array(clicked))\n",
    "        return pred_lst"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Batch    0/15625   mean click = 0.23291435837745667\n",
      "Batch   25/15625   mean click = 0.2441655397415161\n",
      "Batch   50/15625   mean click = 0.20712077617645264\n",
      "Batch   75/15625   mean click = 0.22837260365486145\n",
      "Batch  100/15625   mean click = 0.23258405923843384\n",
      "Batch  125/15625   mean click = 0.21670520305633545\n",
      "Batch  150/15625   mean click = 0.17298942804336548\n",
      "Batch  175/15625   mean click = 0.23729506134986877\n",
      "Batch  200/15625   mean click = 0.2072361707687378\n",
      "Batch  225/15625   mean click = 0.22949182987213135\n",
      "Batch  250/15625   mean click = 0.191495880484581\n",
      "Batch  275/15625   mean click = 0.19321107864379883\n",
      "Batch  300/15625   mean click = 0.26005569100379944\n",
      "Batch  325/15625   mean click = 0.2759130895137787\n",
      "Batch  350/15625   mean click = 0.20853781700134277\n",
      "Batch  375/15625   mean click = 0.21732787787914276\n",
      "Batch  400/15625   mean click = 0.24034711718559265\n",
      "Batch  425/15625   mean click = 0.23197221755981445\n",
      "Batch  450/15625   mean click = 0.2815670669078827\n",
      "Batch  475/15625   mean click = 0.2584182620048523\n",
      "Batch  500/15625   mean click = 0.21042156219482422\n",
      "Batch  525/15625   mean click = 0.22141112387180328\n",
      "Batch  550/15625   mean click = 0.25858086347579956\n",
      "Batch  575/15625   mean click = 0.20956207811832428\n",
      "Batch  600/15625   mean click = 0.23128065466880798\n",
      "Batch  625/15625   mean click = 0.24947309494018555\n",
      "Batch  650/15625   mean click = 0.24460802972316742\n",
      "Batch  675/15625   mean click = 0.2240004986524582\n",
      "Batch  700/15625   mean click = 0.2507709860801697\n",
      "Batch  725/15625   mean click = 0.20248760282993317\n",
      "Batch  750/15625   mean click = 0.24853116273880005\n",
      "Batch  775/15625   mean click = 0.2915969789028168\n",
      "Batch  800/15625   mean click = 0.2404537796974182\n",
      "Batch  825/15625   mean click = 0.21902874112129211\n",
      "Batch  850/15625   mean click = 0.30120936036109924\n",
      "Batch  875/15625   mean click = 0.2606097161769867\n",
      "Batch  900/15625   mean click = 0.2231488823890686\n",
      "Batch  925/15625   mean click = 0.24501827359199524\n",
      "Batch  950/15625   mean click = 0.249719500541687\n",
      "Batch  975/15625   mean click = 0.2848314046859741\n",
      "Batch 1000/15625   mean click = 0.23386362195014954\n",
      "Batch 1025/15625   mean click = 0.21629862487316132\n",
      "Batch 1050/15625   mean click = 0.2658974528312683\n",
      "Batch 1075/15625   mean click = 0.21902355551719666\n",
      "Batch 1100/15625   mean click = 0.2627113163471222\n",
      "Batch 1125/15625   mean click = 0.19506889581680298\n",
      "Batch 1150/15625   mean click = 0.2193523645401001\n",
      "Batch 1175/15625   mean click = 0.23418831825256348\n",
      "Batch 1200/15625   mean click = 0.22018036246299744\n",
      "Batch 1225/15625   mean click = 0.2477821558713913\n",
      "Batch 1250/15625   mean click = 0.21504858136177063\n",
      "Batch 1275/15625   mean click = 0.21435603499412537\n",
      "Batch 1300/15625   mean click = 0.20656633377075195\n",
      "Batch 1325/15625   mean click = 0.2462851107120514\n",
      "Batch 1350/15625   mean click = 0.24692277610301971\n",
      "Batch 1375/15625   mean click = 0.23033024370670319\n",
      "Batch 1400/15625   mean click = 0.2864271402359009\n",
      "Batch 1425/15625   mean click = 0.19320952892303467\n",
      "Batch 1450/15625   mean click = 0.21468305587768555\n",
      "Batch 1475/15625   mean click = 0.22578831017017365\n",
      "Batch 1500/15625   mean click = 0.30481821298599243\n",
      "Batch 1525/15625   mean click = 0.24942845106124878\n",
      "Batch 1550/15625   mean click = 0.2640255093574524\n",
      "Batch 1575/15625   mean click = 0.2613033950328827\n",
      "Batch 1600/15625   mean click = 0.24728131294250488\n",
      "Batch 1625/15625   mean click = 0.262564480304718\n",
      "Batch 1650/15625   mean click = 0.30527234077453613\n",
      "Batch 1675/15625   mean click = 0.2569473683834076\n",
      "Batch 1700/15625   mean click = 0.29888713359832764\n",
      "Batch 1725/15625   mean click = 0.24160954356193542\n",
      "Batch 1750/15625   mean click = 0.2408321648836136\n",
      "Batch 1775/15625   mean click = 0.2679857909679413\n",
      "Batch 1800/15625   mean click = 0.26932618021965027\n",
      "Batch 1825/15625   mean click = 0.24404878914356232\n",
      "Batch 1850/15625   mean click = 0.3149767220020294\n",
      "Batch 1875/15625   mean click = 0.22843793034553528\n",
      "Batch 1900/15625   mean click = 0.3150481581687927\n",
      "Batch 1925/15625   mean click = 0.24101139605045319\n",
      "Batch 1950/15625   mean click = 0.2482905089855194\n",
      "Batch 1975/15625   mean click = 0.30278003215789795\n",
      "Batch 2000/15625   mean click = 0.29524630308151245\n",
      "Batch 2025/15625   mean click = 0.3220912218093872\n",
      "Batch 2050/15625   mean click = 0.2532612085342407\n",
      "Batch 2075/15625   mean click = 0.2671999931335449\n",
      "Batch 2100/15625   mean click = 0.28794556856155396\n",
      "Batch 2125/15625   mean click = 0.2709619700908661\n",
      "Batch 2150/15625   mean click = 0.28141435980796814\n",
      "Batch 2175/15625   mean click = 0.2454085648059845\n",
      "Batch 2200/15625   mean click = 0.2713252902030945\n",
      "Batch 2225/15625   mean click = 0.2592805325984955\n",
      "Batch 2250/15625   mean click = 0.24157479405403137\n",
      "Batch 2275/15625   mean click = 0.2970731258392334\n",
      "Batch 2300/15625   mean click = 0.3063781261444092\n",
      "Batch 2325/15625   mean click = 0.2986487150192261\n",
      "Batch 2350/15625   mean click = 0.2926645874977112\n",
      "Batch 2375/15625   mean click = 0.2780248522758484\n",
      "Batch 2400/15625   mean click = 0.24444299936294556\n",
      "Batch 2425/15625   mean click = 0.2800978422164917\n",
      "Batch 2450/15625   mean click = 0.22745941579341888\n",
      "Batch 2475/15625   mean click = 0.3161911964416504\n",
      "Batch 2500/15625   mean click = 0.2752346098423004\n",
      "Batch 2525/15625   mean click = 0.25697511434555054\n",
      "Batch 2550/15625   mean click = 0.2913321554660797\n",
      "Batch 2575/15625   mean click = 0.2886938452720642\n",
      "Batch 2600/15625   mean click = 0.28389665484428406\n",
      "Batch 2625/15625   mean click = 0.27714526653289795\n",
      "Batch 2650/15625   mean click = 0.2934851050376892\n",
      "Batch 2675/15625   mean click = 0.2713155150413513\n",
      "Batch 2700/15625   mean click = 0.23217758536338806\n",
      "Batch 2725/15625   mean click = 0.2850233316421509\n",
      "Batch 2750/15625   mean click = 0.285381019115448\n",
      "Batch 2775/15625   mean click = 0.26818883419036865\n",
      "Batch 2800/15625   mean click = 0.27579212188720703\n",
      "Batch 2825/15625   mean click = 0.3016214370727539\n",
      "Batch 2850/15625   mean click = 0.2805367708206177\n",
      "Batch 2875/15625   mean click = 0.3095439672470093\n",
      "Batch 2900/15625   mean click = 0.2553825378417969\n",
      "Batch 2925/15625   mean click = 0.30087631940841675\n",
      "Batch 2950/15625   mean click = 0.2662791907787323\n",
      "Batch 2975/15625   mean click = 0.27591174840927124\n",
      "Batch 3000/15625   mean click = 0.29403218626976013\n",
      "Batch 3025/15625   mean click = 0.29006248712539673\n",
      "Batch 3050/15625   mean click = 0.30392149090766907\n",
      "Batch 3075/15625   mean click = 0.27099326252937317\n",
      "Batch 3100/15625   mean click = 0.2485162764787674\n",
      "Batch 3125/15625   mean click = 0.305447518825531\n",
      "Batch 3150/15625   mean click = 0.3219958543777466\n",
      "Batch 3175/15625   mean click = 0.30341118574142456\n",
      "Batch 3200/15625   mean click = 0.23334601521492004\n",
      "Batch 3225/15625   mean click = 0.2623499035835266\n",
      "Batch 3250/15625   mean click = 0.23509493470191956\n",
      "Batch 3275/15625   mean click = 0.2673075795173645\n",
      "Batch 3300/15625   mean click = 0.2538890242576599\n",
      "Batch 3325/15625   mean click = 0.27901899814605713\n",
      "Batch 3350/15625   mean click = 0.3135005533695221\n",
      "Batch 3375/15625   mean click = 0.28049951791763306\n",
      "Batch 3400/15625   mean click = 0.25516021251678467\n",
      "Batch 3425/15625   mean click = 0.28006017208099365\n",
      "Batch 3450/15625   mean click = 0.3007710576057434\n",
      "Batch 3475/15625   mean click = 0.3004833459854126\n",
      "Batch 3500/15625   mean click = 0.27091512084007263\n",
      "Batch 3525/15625   mean click = 0.2840121388435364\n",
      "Batch 3550/15625   mean click = 0.2548440992832184\n",
      "Batch 3575/15625   mean click = 0.27382415533065796\n",
      "Batch 3600/15625   mean click = 0.28075408935546875\n",
      "Batch 3625/15625   mean click = 0.2712467312812805\n",
      "Batch 3650/15625   mean click = 0.31362423300743103\n",
      "Batch 3675/15625   mean click = 0.227085679769516\n",
      "Batch 3700/15625   mean click = 0.31734350323677063\n",
      "Batch 3725/15625   mean click = 0.3097921311855316\n",
      "Batch 3750/15625   mean click = 0.31607067584991455\n",
      "Batch 3775/15625   mean click = 0.29306477308273315\n",
      "Batch 3800/15625   mean click = 0.24355685710906982\n",
      "Batch 3825/15625   mean click = 0.28386977314949036\n",
      "Batch 3850/15625   mean click = 0.2939763069152832\n",
      "Batch 3875/15625   mean click = 0.23161396384239197\n",
      "Batch 3900/15625   mean click = 0.3062880337238312\n",
      "Batch 3925/15625   mean click = 0.2720099687576294\n",
      "Batch 3950/15625   mean click = 0.2696394622325897\n",
      "Batch 3975/15625   mean click = 0.2803455591201782\n",
      "Batch 4000/15625   mean click = 0.2791885733604431\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Batch 4025/15625   mean click = 0.25718551874160767\n",
      "Batch 4050/15625   mean click = 0.22358953952789307\n",
      "Batch 4075/15625   mean click = 0.22339928150177002\n",
      "Batch 4100/15625   mean click = 0.30588382482528687\n",
      "Batch 4125/15625   mean click = 0.2960987091064453\n",
      "Batch 4150/15625   mean click = 0.27914974093437195\n",
      "Batch 4175/15625   mean click = 0.26086509227752686\n",
      "Batch 4200/15625   mean click = 0.32542452216148376\n",
      "Batch 4225/15625   mean click = 0.29780733585357666\n",
      "Batch 4250/15625   mean click = 0.287104994058609\n",
      "Batch 4275/15625   mean click = 0.23420193791389465\n",
      "Batch 4300/15625   mean click = 0.3061618208885193\n",
      "Batch 4325/15625   mean click = 0.2912747263908386\n",
      "Batch 4350/15625   mean click = 0.29544275999069214\n",
      "Batch 4375/15625   mean click = 0.32753801345825195\n",
      "Batch 4400/15625   mean click = 0.3233729898929596\n",
      "Batch 4425/15625   mean click = 0.3018587827682495\n",
      "Batch 4450/15625   mean click = 0.33128368854522705\n",
      "Batch 4475/15625   mean click = 0.2456045001745224\n",
      "Batch 4500/15625   mean click = 0.2169981598854065\n",
      "Batch 4525/15625   mean click = 0.23826102912425995\n",
      "Batch 4550/15625   mean click = 0.3258874714374542\n",
      "Batch 4575/15625   mean click = 0.2524356245994568\n",
      "Batch 4600/15625   mean click = 0.2741822302341461\n",
      "Batch 4625/15625   mean click = 0.27673596143722534\n",
      "Batch 4650/15625   mean click = 0.31900009512901306\n",
      "Batch 4675/15625   mean click = 0.2568429112434387\n",
      "Batch 4700/15625   mean click = 0.285392701625824\n",
      "Batch 4725/15625   mean click = 0.24406886100769043\n",
      "Batch 4750/15625   mean click = 0.2348766326904297\n",
      "Batch 4775/15625   mean click = 0.26249420642852783\n",
      "Batch 4800/15625   mean click = 0.3017648160457611\n",
      "Batch 4825/15625   mean click = 0.2542944550514221\n",
      "Batch 4850/15625   mean click = 0.2977829575538635\n",
      "Batch 4875/15625   mean click = 0.25545716285705566\n",
      "Batch 4900/15625   mean click = 0.3064119815826416\n",
      "Batch 4925/15625   mean click = 0.25934329628944397\n",
      "Batch 4950/15625   mean click = 0.27419453859329224\n",
      "Batch 4975/15625   mean click = 0.2775963544845581\n",
      "Batch 5000/15625   mean click = 0.27371394634246826\n",
      "Batch 5025/15625   mean click = 0.27172034978866577\n",
      "Batch 5050/15625   mean click = 0.2634974718093872\n",
      "Batch 5075/15625   mean click = 0.32296162843704224\n",
      "Batch 5100/15625   mean click = 0.306207537651062\n",
      "Batch 5125/15625   mean click = 0.25057852268218994\n",
      "Batch 5150/15625   mean click = 0.2762766480445862\n",
      "Batch 5175/15625   mean click = 0.2627096176147461\n",
      "Batch 5200/15625   mean click = 0.3174428641796112\n",
      "Batch 5225/15625   mean click = 0.2435666173696518\n",
      "Batch 5250/15625   mean click = 0.2773308753967285\n",
      "Batch 5275/15625   mean click = 0.30101436376571655\n",
      "Batch 5300/15625   mean click = 0.2728602886199951\n",
      "Batch 5325/15625   mean click = 0.2913566827774048\n",
      "Batch 5350/15625   mean click = 0.2724256217479706\n",
      "Batch 5375/15625   mean click = 0.30804532766342163\n",
      "Batch 5400/15625   mean click = 0.2751378118991852\n",
      "Batch 5425/15625   mean click = 0.2517496645450592\n",
      "Batch 5450/15625   mean click = 0.28921374678611755\n",
      "Batch 5475/15625   mean click = 0.292826771736145\n",
      "Batch 5500/15625   mean click = 0.28025132417678833\n",
      "Batch 5525/15625   mean click = 0.23521772027015686\n",
      "Batch 5550/15625   mean click = 0.23163796961307526\n",
      "Batch 5575/15625   mean click = 0.2866138219833374\n",
      "Batch 5600/15625   mean click = 0.2738368511199951\n",
      "Batch 5625/15625   mean click = 0.31272318959236145\n",
      "Batch 5650/15625   mean click = 0.2576949894428253\n",
      "Batch 5675/15625   mean click = 0.3073650002479553\n",
      "Batch 5700/15625   mean click = 0.2694709002971649\n",
      "Batch 5725/15625   mean click = 0.21096189320087433\n",
      "Batch 5750/15625   mean click = 0.27276310324668884\n",
      "Batch 5775/15625   mean click = 0.2686390280723572\n",
      "Batch 5800/15625   mean click = 0.32414552569389343\n",
      "Batch 5825/15625   mean click = 0.30844646692276\n",
      "Batch 5850/15625   mean click = 0.19926728308200836\n",
      "Batch 5875/15625   mean click = 0.2810819745063782\n",
      "Batch 5900/15625   mean click = 0.2643240690231323\n",
      "Batch 5925/15625   mean click = 0.2625789940357208\n",
      "Batch 5950/15625   mean click = 0.2598344087600708\n",
      "Batch 5975/15625   mean click = 0.3022887110710144\n",
      "Batch 6000/15625   mean click = 0.31881752610206604\n",
      "Batch 6025/15625   mean click = 0.2676700949668884\n",
      "Batch 6050/15625   mean click = 0.2880017161369324\n",
      "Batch 6075/15625   mean click = 0.2524959444999695\n",
      "Batch 6100/15625   mean click = 0.250789612531662\n",
      "Batch 6125/15625   mean click = 0.2646103501319885\n",
      "Batch 6150/15625   mean click = 0.2563849091529846\n",
      "Batch 6175/15625   mean click = 0.31899335980415344\n",
      "Batch 6200/15625   mean click = 0.281768798828125\n",
      "Batch 6225/15625   mean click = 0.3199000954627991\n",
      "Batch 6250/15625   mean click = 0.2614053785800934\n",
      "Batch 6275/15625   mean click = 0.2604416310787201\n",
      "Batch 6300/15625   mean click = 0.26160117983818054\n",
      "Batch 6325/15625   mean click = 0.26663458347320557\n",
      "Batch 6350/15625   mean click = 0.2753901481628418\n",
      "Batch 6375/15625   mean click = 0.2558828592300415\n",
      "Batch 6400/15625   mean click = 0.2775030732154846\n",
      "Batch 6425/15625   mean click = 0.2736002206802368\n",
      "Batch 6450/15625   mean click = 0.24347063899040222\n",
      "Batch 6475/15625   mean click = 0.27101755142211914\n",
      "Batch 6500/15625   mean click = 0.32881301641464233\n",
      "Batch 6525/15625   mean click = 0.25715577602386475\n",
      "Batch 6550/15625   mean click = 0.3208048939704895\n",
      "Batch 6575/15625   mean click = 0.3033876419067383\n",
      "Batch 6600/15625   mean click = 0.30061304569244385\n",
      "Batch 6625/15625   mean click = 0.2623308598995209\n",
      "Batch 6650/15625   mean click = 0.260341614484787\n",
      "Batch 6675/15625   mean click = 0.25848573446273804\n",
      "Batch 6700/15625   mean click = 0.26607441902160645\n",
      "Batch 6725/15625   mean click = 0.2698628306388855\n",
      "Batch 6750/15625   mean click = 0.2455660104751587\n",
      "Batch 6775/15625   mean click = 0.29036104679107666\n",
      "Batch 6800/15625   mean click = 0.285086989402771\n",
      "Batch 6825/15625   mean click = 0.21151268482208252\n",
      "Batch 6850/15625   mean click = 0.2655462622642517\n",
      "Batch 6875/15625   mean click = 0.24644452333450317\n",
      "Batch 6900/15625   mean click = 0.23762556910514832\n",
      "Batch 6925/15625   mean click = 0.2715938091278076\n",
      "Batch 6950/15625   mean click = 0.2973743677139282\n",
      "Batch 6975/15625   mean click = 0.29023924469947815\n",
      "Batch 7000/15625   mean click = 0.2907595932483673\n",
      "Batch 7025/15625   mean click = 0.27066850662231445\n",
      "Batch 7050/15625   mean click = 0.28242790699005127\n",
      "Batch 7075/15625   mean click = 0.3079938292503357\n",
      "Batch 7100/15625   mean click = 0.28578487038612366\n",
      "Batch 7125/15625   mean click = 0.2438366860151291\n",
      "Batch 7150/15625   mean click = 0.2871744632720947\n",
      "Batch 7175/15625   mean click = 0.2882387936115265\n",
      "Batch 7200/15625   mean click = 0.2623291313648224\n",
      "Batch 7225/15625   mean click = 0.2518889307975769\n",
      "Batch 7250/15625   mean click = 0.27199363708496094\n",
      "Batch 7275/15625   mean click = 0.30084288120269775\n",
      "Batch 7300/15625   mean click = 0.266970157623291\n",
      "Batch 7325/15625   mean click = 0.32472026348114014\n",
      "Batch 7350/15625   mean click = 0.3199840784072876\n",
      "Batch 7375/15625   mean click = 0.27615681290626526\n",
      "Batch 7400/15625   mean click = 0.2844695746898651\n",
      "Batch 7425/15625   mean click = 0.2883554995059967\n",
      "Batch 7450/15625   mean click = 0.3241155445575714\n",
      "Batch 7475/15625   mean click = 0.26166850328445435\n",
      "Batch 7500/15625   mean click = 0.3061290383338928\n",
      "Batch 7525/15625   mean click = 0.2761879861354828\n",
      "Batch 7550/15625   mean click = 0.2523140609264374\n",
      "Batch 7575/15625   mean click = 0.3251604437828064\n",
      "Batch 7600/15625   mean click = 0.29608020186424255\n",
      "Batch 7625/15625   mean click = 0.26103585958480835\n",
      "Batch 7650/15625   mean click = 0.3056957721710205\n",
      "Batch 7675/15625   mean click = 0.30553168058395386\n",
      "Batch 7700/15625   mean click = 0.3113921880722046\n",
      "Batch 7725/15625   mean click = 0.2847326397895813\n",
      "Batch 7750/15625   mean click = 0.26855871081352234\n",
      "Batch 7775/15625   mean click = 0.26759976148605347\n",
      "Batch 7800/15625   mean click = 0.2419605851173401\n",
      "Batch 7825/15625   mean click = 0.25843745470046997\n",
      "Batch 7850/15625   mean click = 0.277112752199173\n",
      "Batch 7875/15625   mean click = 0.308706134557724\n",
      "Batch 7900/15625   mean click = 0.2924591302871704\n",
      "Batch 7925/15625   mean click = 0.2692617177963257\n",
      "Batch 7950/15625   mean click = 0.2641686499118805\n",
      "Batch 7975/15625   mean click = 0.2997850179672241\n",
      "Batch 8000/15625   mean click = 0.2838894724845886\n",
      "Batch 8025/15625   mean click = 0.321979820728302\n",
      "Batch 8050/15625   mean click = 0.3067566752433777\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Batch 8075/15625   mean click = 0.2645380198955536\n",
      "Batch 8100/15625   mean click = 0.24229785799980164\n",
      "Batch 8125/15625   mean click = 0.2698376178741455\n",
      "Batch 8150/15625   mean click = 0.2508085072040558\n",
      "Batch 8175/15625   mean click = 0.30194994807243347\n",
      "Batch 8200/15625   mean click = 0.27145373821258545\n",
      "Batch 8225/15625   mean click = 0.2738555073738098\n",
      "Batch 8250/15625   mean click = 0.3001673221588135\n",
      "Batch 8275/15625   mean click = 0.26563286781311035\n",
      "Batch 8300/15625   mean click = 0.2813803553581238\n",
      "Batch 8325/15625   mean click = 0.24460750818252563\n",
      "Batch 8350/15625   mean click = 0.24737372994422913\n",
      "Batch 8375/15625   mean click = 0.3074081540107727\n",
      "Batch 8400/15625   mean click = 0.2729794383049011\n",
      "Batch 8425/15625   mean click = 0.2626821994781494\n",
      "Batch 8450/15625   mean click = 0.22110632061958313\n",
      "Batch 8475/15625   mean click = 0.21573123335838318\n",
      "Batch 8500/15625   mean click = 0.2818642258644104\n",
      "Batch 8525/15625   mean click = 0.2870376408100128\n",
      "Batch 8550/15625   mean click = 0.22428098320960999\n",
      "Batch 8575/15625   mean click = 0.22631293535232544\n",
      "Batch 8600/15625   mean click = 0.2478935271501541\n",
      "Batch 8625/15625   mean click = 0.32209333777427673\n",
      "Batch 8650/15625   mean click = 0.293104887008667\n",
      "Batch 8675/15625   mean click = 0.23455239832401276\n",
      "Batch 8700/15625   mean click = 0.22833220660686493\n",
      "Batch 8725/15625   mean click = 0.21496166288852692\n",
      "Batch 8750/15625   mean click = 0.24974845349788666\n",
      "Batch 8775/15625   mean click = 0.27586817741394043\n",
      "Batch 8800/15625   mean click = 0.306018203496933\n",
      "Batch 8825/15625   mean click = 0.2460077404975891\n",
      "Batch 8850/15625   mean click = 0.21610261499881744\n",
      "Batch 8875/15625   mean click = 0.2908136248588562\n",
      "Batch 8900/15625   mean click = 0.27408134937286377\n",
      "Batch 8925/15625   mean click = 0.23393768072128296\n",
      "Batch 8950/15625   mean click = 0.2339419424533844\n",
      "Batch 8975/15625   mean click = 0.257934033870697\n",
      "Batch 9000/15625   mean click = 0.33079469203948975\n",
      "Batch 9025/15625   mean click = 0.27227655053138733\n",
      "Batch 9050/15625   mean click = 0.2383759319782257\n",
      "Batch 9075/15625   mean click = 0.2413852959871292\n",
      "Batch 9100/15625   mean click = 0.2742912173271179\n",
      "Batch 9125/15625   mean click = 0.29012101888656616\n",
      "Batch 9150/15625   mean click = 0.2866859436035156\n",
      "Batch 9175/15625   mean click = 0.23866719007492065\n",
      "Batch 9200/15625   mean click = 0.30420732498168945\n",
      "Batch 9225/15625   mean click = 0.25872769951820374\n",
      "Batch 9250/15625   mean click = 0.2772880792617798\n",
      "Batch 9275/15625   mean click = 0.25918954610824585\n",
      "Batch 9300/15625   mean click = 0.29246097803115845\n",
      "Batch 9325/15625   mean click = 0.32306134700775146\n",
      "Batch 9350/15625   mean click = 0.2740115523338318\n",
      "Batch 9375/15625   mean click = 0.2666374444961548\n",
      "Batch 9400/15625   mean click = 0.25345319509506226\n",
      "Batch 9425/15625   mean click = 0.29463714361190796\n",
      "Batch 9450/15625   mean click = 0.2890479862689972\n",
      "Batch 9475/15625   mean click = 0.3056392967700958\n",
      "Batch 9500/15625   mean click = 0.2560412883758545\n",
      "Batch 9525/15625   mean click = 0.25405964255332947\n",
      "Batch 9550/15625   mean click = 0.2402852326631546\n",
      "Batch 9575/15625   mean click = 0.2491203248500824\n",
      "Batch 9600/15625   mean click = 0.2778374254703522\n",
      "Batch 9625/15625   mean click = 0.27804601192474365\n",
      "Batch 9650/15625   mean click = 0.26543128490448\n",
      "Batch 9675/15625   mean click = 0.3012971878051758\n",
      "Batch 9700/15625   mean click = 0.2651318609714508\n",
      "Batch 9725/15625   mean click = 0.28060853481292725\n",
      "Batch 9750/15625   mean click = 0.25769922137260437\n",
      "Batch 9775/15625   mean click = 0.2545686364173889\n",
      "Batch 9800/15625   mean click = 0.31317371129989624\n",
      "Batch 9825/15625   mean click = 0.2795979082584381\n",
      "Batch 9850/15625   mean click = 0.30418023467063904\n",
      "Batch 9875/15625   mean click = 0.22264662384986877\n",
      "Batch 9900/15625   mean click = 0.26135411858558655\n",
      "Batch 9925/15625   mean click = 0.27067098021507263\n",
      "Batch 9950/15625   mean click = 0.2868715524673462\n",
      "Batch 9975/15625   mean click = 0.2901080846786499\n",
      "Batch 10000/15625   mean click = 0.23502004146575928\n",
      "Batch 10025/15625   mean click = 0.320210337638855\n",
      "Batch 10050/15625   mean click = 0.2630918025970459\n",
      "Batch 10075/15625   mean click = 0.2309572398662567\n",
      "Batch 10100/15625   mean click = 0.22893229126930237\n",
      "Batch 10125/15625   mean click = 0.31690874695777893\n",
      "Batch 10150/15625   mean click = 0.2865493893623352\n",
      "Batch 10175/15625   mean click = 0.2718725800514221\n",
      "Batch 10200/15625   mean click = 0.2840577960014343\n",
      "Batch 10225/15625   mean click = 0.23264998197555542\n",
      "Batch 10250/15625   mean click = 0.2572557330131531\n",
      "Batch 10275/15625   mean click = 0.26509109139442444\n",
      "Batch 10300/15625   mean click = 0.2605787515640259\n",
      "Batch 10325/15625   mean click = 0.24539488554000854\n",
      "Batch 10350/15625   mean click = 0.2791145443916321\n",
      "Batch 10375/15625   mean click = 0.28545454144477844\n",
      "Batch 10400/15625   mean click = 0.23496991395950317\n",
      "Batch 10425/15625   mean click = 0.27062636613845825\n",
      "Batch 10450/15625   mean click = 0.3185259997844696\n",
      "Batch 10475/15625   mean click = 0.25554707646369934\n",
      "Batch 10500/15625   mean click = 0.25564101338386536\n",
      "Batch 10525/15625   mean click = 0.3094778060913086\n",
      "Batch 10550/15625   mean click = 0.2502489686012268\n",
      "Batch 10575/15625   mean click = 0.3003029525279999\n",
      "Batch 10600/15625   mean click = 0.23182345926761627\n",
      "Batch 10625/15625   mean click = 0.27443206310272217\n",
      "Batch 10650/15625   mean click = 0.28042465448379517\n",
      "Batch 10675/15625   mean click = 0.2827758193016052\n",
      "Batch 10700/15625   mean click = 0.24340704083442688\n",
      "Batch 10725/15625   mean click = 0.26848310232162476\n",
      "Batch 10750/15625   mean click = 0.2561873495578766\n",
      "Batch 10775/15625   mean click = 0.2662402391433716\n",
      "Batch 10800/15625   mean click = 0.27087903022766113\n",
      "Batch 10825/15625   mean click = 0.2936164140701294\n",
      "Batch 10850/15625   mean click = 0.30190062522888184\n",
      "Batch 10875/15625   mean click = 0.24953627586364746\n",
      "Batch 10900/15625   mean click = 0.2909058928489685\n",
      "Batch 10925/15625   mean click = 0.306085467338562\n",
      "Batch 10950/15625   mean click = 0.24082092940807343\n",
      "Batch 10975/15625   mean click = 0.3059123754501343\n",
      "Batch 11000/15625   mean click = 0.29687535762786865\n",
      "Batch 11025/15625   mean click = 0.25383126735687256\n",
      "Batch 11050/15625   mean click = 0.24594710767269135\n",
      "Batch 11075/15625   mean click = 0.25015905499458313\n",
      "Batch 11100/15625   mean click = 0.27861613035202026\n",
      "Batch 11125/15625   mean click = 0.2425440102815628\n",
      "Batch 11150/15625   mean click = 0.27809780836105347\n",
      "Batch 11175/15625   mean click = 0.24592047929763794\n",
      "Batch 11200/15625   mean click = 0.25939035415649414\n",
      "Batch 11225/15625   mean click = 0.25774189829826355\n",
      "Batch 11250/15625   mean click = 0.26196348667144775\n",
      "Batch 11275/15625   mean click = 0.2666991055011749\n",
      "Batch 11300/15625   mean click = 0.24260637164115906\n",
      "Batch 11325/15625   mean click = 0.22068387269973755\n",
      "Batch 11350/15625   mean click = 0.26473021507263184\n",
      "Batch 11375/15625   mean click = 0.24366328120231628\n",
      "Batch 11400/15625   mean click = 0.28523504734039307\n",
      "Batch 11425/15625   mean click = 0.2591107189655304\n",
      "Batch 11450/15625   mean click = 0.27948763966560364\n",
      "Batch 11475/15625   mean click = 0.29005953669548035\n",
      "Batch 11500/15625   mean click = 0.273872435092926\n",
      "Batch 11525/15625   mean click = 0.2709029018878937\n",
      "Batch 11550/15625   mean click = 0.26862919330596924\n",
      "Batch 11575/15625   mean click = 0.25630396604537964\n",
      "Batch 11600/15625   mean click = 0.262204647064209\n",
      "Batch 11625/15625   mean click = 0.2838139533996582\n",
      "Batch 11650/15625   mean click = 0.3076741099357605\n",
      "Batch 11675/15625   mean click = 0.29361075162887573\n",
      "Batch 11700/15625   mean click = 0.3019731640815735\n",
      "Batch 11725/15625   mean click = 0.2929476201534271\n",
      "Batch 11750/15625   mean click = 0.28845199942588806\n",
      "Batch 11775/15625   mean click = 0.30240631103515625\n",
      "Batch 11800/15625   mean click = 0.2637265622615814\n",
      "Batch 11825/15625   mean click = 0.24628281593322754\n",
      "Batch 11850/15625   mean click = 0.3446846902370453\n",
      "Batch 11875/15625   mean click = 0.2371024191379547\n",
      "Batch 11900/15625   mean click = 0.3066999614238739\n",
      "Batch 11925/15625   mean click = 0.25225499272346497\n",
      "Batch 11950/15625   mean click = 0.27281612157821655\n",
      "Batch 11975/15625   mean click = 0.272046834230423\n",
      "Batch 12000/15625   mean click = 0.2591683864593506\n",
      "Batch 12025/15625   mean click = 0.29305604100227356\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Batch 12050/15625   mean click = 0.34219467639923096\n",
      "Batch 12075/15625   mean click = 0.266782283782959\n",
      "Batch 12100/15625   mean click = 0.2693360447883606\n",
      "Batch 12125/15625   mean click = 0.2514916658401489\n",
      "Batch 12150/15625   mean click = 0.25578930974006653\n",
      "Batch 12175/15625   mean click = 0.2604748010635376\n",
      "Batch 12200/15625   mean click = 0.29582804441452026\n",
      "Batch 12225/15625   mean click = 0.24068963527679443\n",
      "Batch 12250/15625   mean click = 0.29262226819992065\n",
      "Batch 12275/15625   mean click = 0.26445308327674866\n",
      "Batch 12300/15625   mean click = 0.25775599479675293\n",
      "Batch 12325/15625   mean click = 0.27819424867630005\n",
      "Batch 12350/15625   mean click = 0.28408730030059814\n",
      "Batch 12375/15625   mean click = 0.27631819248199463\n",
      "Batch 12400/15625   mean click = 0.2731817960739136\n",
      "Batch 12425/15625   mean click = 0.22351673245429993\n",
      "Batch 12450/15625   mean click = 0.26238077878952026\n",
      "Batch 12475/15625   mean click = 0.32303160429000854\n",
      "Batch 12500/15625   mean click = 0.2642633318901062\n",
      "Batch 12525/15625   mean click = 0.2618941366672516\n",
      "Batch 12550/15625   mean click = 0.25486546754837036\n",
      "Batch 12575/15625   mean click = 0.27798259258270264\n",
      "Batch 12600/15625   mean click = 0.25364628434181213\n",
      "Batch 12625/15625   mean click = 0.25109636783599854\n",
      "Batch 12650/15625   mean click = 0.24437829852104187\n",
      "Batch 12675/15625   mean click = 0.2651756703853607\n",
      "Batch 12700/15625   mean click = 0.22886261343955994\n",
      "Batch 12725/15625   mean click = 0.23201921582221985\n",
      "Batch 12750/15625   mean click = 0.24216631054878235\n",
      "Batch 12775/15625   mean click = 0.3082335591316223\n",
      "Batch 12800/15625   mean click = 0.2592727541923523\n",
      "Batch 12825/15625   mean click = 0.23861363530158997\n",
      "Batch 12850/15625   mean click = 0.2709859013557434\n",
      "Batch 12875/15625   mean click = 0.2698098421096802\n",
      "Batch 12900/15625   mean click = 0.28476250171661377\n",
      "Batch 12925/15625   mean click = 0.2557106614112854\n",
      "Batch 12950/15625   mean click = 0.2652107775211334\n",
      "Batch 12975/15625   mean click = 0.27283746004104614\n",
      "Batch 13000/15625   mean click = 0.2858549654483795\n",
      "Batch 13025/15625   mean click = 0.25081372261047363\n",
      "Batch 13050/15625   mean click = 0.24949879944324493\n",
      "Batch 13075/15625   mean click = 0.26654285192489624\n",
      "Batch 13100/15625   mean click = 0.28483426570892334\n",
      "Batch 13125/15625   mean click = 0.3206072449684143\n",
      "Batch 13150/15625   mean click = 0.2390439361333847\n",
      "Batch 13175/15625   mean click = 0.3260338008403778\n",
      "Batch 13200/15625   mean click = 0.24697738885879517\n",
      "Batch 13225/15625   mean click = 0.249964639544487\n",
      "Batch 13250/15625   mean click = 0.2770872414112091\n",
      "Batch 13275/15625   mean click = 0.2598469853401184\n",
      "Batch 13300/15625   mean click = 0.23169907927513123\n",
      "Batch 13325/15625   mean click = 0.22219999134540558\n",
      "Batch 13350/15625   mean click = 0.23666644096374512\n",
      "Batch 13375/15625   mean click = 0.2767024338245392\n",
      "Batch 13400/15625   mean click = 0.235416978597641\n",
      "Batch 13425/15625   mean click = 0.263367235660553\n",
      "Batch 13450/15625   mean click = 0.24393127858638763\n",
      "Batch 13475/15625   mean click = 0.24218949675559998\n",
      "Batch 13500/15625   mean click = 0.2734585702419281\n",
      "Batch 13525/15625   mean click = 0.2483057677745819\n",
      "Batch 13550/15625   mean click = 0.23040316998958588\n",
      "Batch 13575/15625   mean click = 0.2484368532896042\n",
      "Batch 13600/15625   mean click = 0.2619830369949341\n",
      "Batch 13625/15625   mean click = 0.2644968628883362\n",
      "Batch 13650/15625   mean click = 0.24982061982154846\n",
      "Batch 13675/15625   mean click = 0.2651306986808777\n",
      "Batch 13700/15625   mean click = 0.30457746982574463\n",
      "Batch 13725/15625   mean click = 0.2550676465034485\n",
      "Batch 13750/15625   mean click = 0.30023810267448425\n",
      "Batch 13775/15625   mean click = 0.29087120294570923\n",
      "Batch 13800/15625   mean click = 0.26569926738739014\n",
      "Batch 13825/15625   mean click = 0.31175708770751953\n",
      "Batch 13850/15625   mean click = 0.2743656635284424\n",
      "Batch 13875/15625   mean click = 0.283373087644577\n",
      "Batch 13900/15625   mean click = 0.32461971044540405\n",
      "Batch 13925/15625   mean click = 0.23345255851745605\n",
      "Batch 13950/15625   mean click = 0.24462054669857025\n",
      "Batch 13975/15625   mean click = 0.2661929726600647\n",
      "Batch 14000/15625   mean click = 0.2709732949733734\n",
      "Batch 14025/15625   mean click = 0.2339239865541458\n",
      "Batch 14050/15625   mean click = 0.2745521664619446\n",
      "Batch 14075/15625   mean click = 0.2514607310295105\n",
      "Batch 14100/15625   mean click = 0.23606665432453156\n",
      "Batch 14125/15625   mean click = 0.28099697828292847\n",
      "Batch 14150/15625   mean click = 0.28107118606567383\n",
      "Batch 14175/15625   mean click = 0.2824988067150116\n",
      "Batch 14200/15625   mean click = 0.299618124961853\n",
      "Batch 14225/15625   mean click = 0.2874676585197449\n",
      "Batch 14250/15625   mean click = 0.2527834177017212\n",
      "Batch 14275/15625   mean click = 0.24541214108467102\n",
      "Batch 14300/15625   mean click = 0.25033870339393616\n",
      "Batch 14325/15625   mean click = 0.3021422028541565\n",
      "Batch 14350/15625   mean click = 0.2616385221481323\n",
      "Batch 14375/15625   mean click = 0.3058599829673767\n",
      "Batch 14400/15625   mean click = 0.25377893447875977\n",
      "Batch 14425/15625   mean click = 0.24613726139068604\n",
      "Batch 14450/15625   mean click = 0.2993311285972595\n",
      "Batch 14475/15625   mean click = 0.26899683475494385\n",
      "Batch 14500/15625   mean click = 0.276201456785202\n",
      "Batch 14525/15625   mean click = 0.24414484202861786\n",
      "Batch 14550/15625   mean click = 0.2923685312271118\n",
      "Batch 14575/15625   mean click = 0.30856239795684814\n",
      "Batch 14600/15625   mean click = 0.2742345631122589\n",
      "Batch 14625/15625   mean click = 0.2956671714782715\n",
      "Batch 14650/15625   mean click = 0.24378985166549683\n",
      "Batch 14675/15625   mean click = 0.29893729090690613\n",
      "Batch 14700/15625   mean click = 0.24327516555786133\n",
      "Batch 14725/15625   mean click = 0.2960813641548157\n",
      "Batch 14750/15625   mean click = 0.27785158157348633\n",
      "Batch 14775/15625   mean click = 0.2644783854484558\n",
      "Batch 14800/15625   mean click = 0.31252849102020264\n",
      "Batch 14825/15625   mean click = 0.22929781675338745\n",
      "Batch 14850/15625   mean click = 0.2979470491409302\n",
      "Batch 14875/15625   mean click = 0.2407722771167755\n",
      "Batch 14900/15625   mean click = 0.2352643609046936\n",
      "Batch 14925/15625   mean click = 0.2965846359729767\n",
      "Batch 14950/15625   mean click = 0.2536908984184265\n",
      "Batch 14975/15625   mean click = 0.2535843253135681\n",
      "Batch 15000/15625   mean click = 0.2520731985569\n",
      "Batch 15025/15625   mean click = 0.2640659213066101\n",
      "Batch 15050/15625   mean click = 0.2623249292373657\n",
      "Batch 15075/15625   mean click = 0.2901681959629059\n",
      "Batch 15100/15625   mean click = 0.2691079080104828\n",
      "Batch 15125/15625   mean click = 0.23739537596702576\n",
      "Batch 15150/15625   mean click = 0.2523791491985321\n",
      "Batch 15175/15625   mean click = 0.26120907068252563\n",
      "Batch 15200/15625   mean click = 0.2321833372116089\n",
      "Batch 15225/15625   mean click = 0.2628520727157593\n",
      "Batch 15250/15625   mean click = 0.25308337807655334\n",
      "Batch 15275/15625   mean click = 0.2616230547428131\n",
      "Batch 15300/15625   mean click = 0.2532472312450409\n",
      "Batch 15325/15625   mean click = 0.24079501628875732\n",
      "Batch 15350/15625   mean click = 0.2832934260368347\n",
      "Batch 15375/15625   mean click = 0.22438381612300873\n",
      "Batch 15400/15625   mean click = 0.251304030418396\n",
      "Batch 15425/15625   mean click = 0.24769312143325806\n",
      "Batch 15450/15625   mean click = 0.3059028387069702\n",
      "Batch 15475/15625   mean click = 0.25334352254867554\n",
      "Batch 15500/15625   mean click = 0.33047187328338623\n",
      "Batch 15525/15625   mean click = 0.3062276840209961\n",
      "Batch 15550/15625   mean click = 0.25956085324287415\n",
      "Batch 15575/15625   mean click = 0.2897615134716034\n",
      "Batch 15600/15625   mean click = 0.22665637731552124\n"
     ]
    }
   ],
   "source": [
    "pred_lst = predict_test(64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.27048749"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(pred_lst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上就是点击率预估的完整过程，没有进行完整数据的训练，并且有很多超参可以调整，从只跑了一次epoch的结果来看，验证集上的LogLoss是0.46，其他数据都在75%~80%之间，这跟FFM、GBDT和FM网络训练的准确率差不多。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 扩展阅读"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " - [`Code for the 3rd place finish for Avazu Click-Through Rate Prediction`](https://github.com/infinitezxc/kaggle-avazu)\n",
    " - [`Kaggle ： Display Advertising Challenge( ctr 预估 )`](http://blog.csdn.net/hero_fantao/article/details/42747281)\n",
    " - [`用机器学习对CTR预估建模`](http://blog.csdn.net/JR_lu/article/details/54836968)\n",
    " - [`Beginner's Guide to Click-Through Rate Prediction with Logistic Regression`](https://turi.com/learn/gallery/notebooks/click_through_rate_prediction_intro.html)\n",
    " - [`2nd place solution for Avazu click-through rate prediction competition`](https://github.com/owenzhang/kaggle-avazu)\n",
    " - [`常见计算广告点击率预估算法总结`](https://zhuanlan.zhihu.com/p/29053940?group_id=916424345212342272)\n",
    " - [`3 Idiots' Approach for Display Advertising Challenge`](https://github.com/guestwalk/kaggle-2014-criteo)\n",
    " - [`Solution to the Outbrain Click Prediction competition`](https://github.com/alexeygrigorev/outbrain-click-prediction-kaggle)\n",
    " - [`Deep Interest Network for Click-Through Rate Prediction`](https://arxiv.org/abs/1706.06978)\n",
    " - [`Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction`](https://arxiv.org/abs/1704.05194)\n",
    " - [`重磅！阿里妈妈首次公开自研CTR预估核心算法MLR`](https://mp.weixin.qq.com/s/MtnHYmPVoDAid9SNHnlzUw)\n",
    " - [`阿里盖坤团队提出深度兴趣网络，更懂用户什么时候会剁手`](http://www.sohu.com/a/152084716_114877)\n",
    " - [`深入FFM原理与实践`](https://tech.meituan.com/deep-understanding-of-ffm-principles-and-practices.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 今天的分享就到这里，就酱~"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
